[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:12:39 dist-test-slave-f7th krb5kdc[29258](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:12:39 dist-test-slave-f7th krb5kdc[29258](info): set up 2 sockets
Aug 11 05:12:39 dist-test-slave-f7th krb5kdc[29258](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 05:12:40.295025 29246 mini_kdc.cc:121] Time spent starting KDC: real 1.214s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Aug 11 05:12:40 dist-test-slave-f7th krb5kdc[29258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:12:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:12:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
I20260811 05:12:40.442919 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:45509
--webserver_interface=127.28.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:46113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:45509
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:40.540360 29284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:40.540606 29284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:40.540683 29284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:40.541952 29284 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:12:40.542004 29284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:40.542023 29284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:40.542040 29284 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:12:40.542054 29284 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:12:40.543684 29284 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:46113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:45509
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:45509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:12:40.543991 29284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:40.544226 29284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:40.549528 29295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:40.549659 29284 server_base.cc:1034] running on GCE node
W20260811 05:12:40.549727 29297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:40.549536 29292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:40.550201 29284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:40.550609 29284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:40.551793 29284 hybrid_clock.cc:648] HybridClock initialized: now 1786425160551742 us; error 68 us; skew 500 ppm
Aug 11 05:12:40 dist-test-slave-f7th krb5kdc[29258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425160, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:40.559108 29284 init.cc:377] Logged in from keytab as kudu/127.28.143.190@KRBTEST.COM (short username kudu)
I20260811 05:12:40.559659 29284 webserver.cc:460] Webserver started at http://127.28.143.190:44833/ using document root <none> and password file <none>
I20260811 05:12:40.559892 29284 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:40.559944 29284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:40.560083 29284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:40.561339 29284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "ec213fae784d4605a42efa03229b3aa2"
format_stamp: "Formatted at 2026-08-11 05:12:40 on dist-test-slave-f7th"
server_key: "46d35304778010b273c310e952a13d55"
server_key_iv: "e1c8ea54d1f53d02f741b7ae7ad4f27d"
server_key_version: "encryptionkey@0"
I20260811 05:12:40.561714 29284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "ec213fae784d4605a42efa03229b3aa2"
format_stamp: "Formatted at 2026-08-11 05:12:40 on dist-test-slave-f7th"
server_key: "46d35304778010b273c310e952a13d55"
server_key_iv: "e1c8ea54d1f53d02f741b7ae7ad4f27d"
server_key_version: "encryptionkey@0"
I20260811 05:12:40.563336 29284 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 05:12:40.564380 29306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:40.564848 29284 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:12:40.564998 29284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "ec213fae784d4605a42efa03229b3aa2"
format_stamp: "Formatted at 2026-08-11 05:12:40 on dist-test-slave-f7th"
server_key: "46d35304778010b273c310e952a13d55"
server_key_iv: "e1c8ea54d1f53d02f741b7ae7ad4f27d"
server_key_version: "encryptionkey@0"
I20260811 05:12:40.565076 29284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:40.577710 29284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:40.580264 29284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:40.580461 29284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:40.584502 29284 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:45509
I20260811 05:12:40.584556 29392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:45509 every 8 connection(s)
I20260811 05:12:40.584964 29284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:12:40.585858 29394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:40.588449 29394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec213fae784d4605a42efa03229b3aa2: Bootstrap starting.
I20260811 05:12:40.589100 29394 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec213fae784d4605a42efa03229b3aa2: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:40.589346 29394 log.cc:824] T 00000000000000000000000000000000 P ec213fae784d4605a42efa03229b3aa2: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:40.590060 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 29284
I20260811 05:12:40.590175 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:12:40.590184 29394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec213fae784d4605a42efa03229b3aa2: No bootstrap required, opened a new log
I20260811 05:12:40.590986 29246 external_mini_cluster.cc:1428] Setting key 6cf9792e5daa3a9859e93ac3788b177f
I20260811 05:12:40.592295 29394 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec213fae784d4605a42efa03229b3aa2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec213fae784d4605a42efa03229b3aa2" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 45509 } }
I20260811 05:12:40.592483 29394 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec213fae784d4605a42efa03229b3aa2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:40.592512 29394 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec213fae784d4605a42efa03229b3aa2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec213fae784d4605a42efa03229b3aa2, State: Initialized, Role: FOLLOWER
I20260811 05:12:40.592669 29394 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec213fae784d4605a42efa03229b3aa2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec213fae784d4605a42efa03229b3aa2" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 45509 } }
I20260811 05:12:40.592746 29394 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec213fae784d4605a42efa03229b3aa2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:12:40.592769 29394 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec213fae784d4605a42efa03229b3aa2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:12:40.592816 29394 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec213fae784d4605a42efa03229b3aa2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:40.593710 29394 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec213fae784d4605a42efa03229b3aa2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec213fae784d4605a42efa03229b3aa2" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 45509 } }
I20260811 05:12:40.593813 29394 leader_election.cc:302] T 00000000000000000000000000000000 P ec213fae784d4605a42efa03229b3aa2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec213fae784d4605a42efa03229b3aa2; no voters:
I20260811 05:12:40.593991 29394 leader_election.cc:288] T 00000000000000000000000000000000 P ec213fae784d4605a42efa03229b3aa2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:12:40.594027 29405 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec213fae784d4605a42efa03229b3aa2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:40.594172 29405 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec213fae784d4605a42efa03229b3aa2 [term 1 LEADER]: Becoming Leader. State: Replica: ec213fae784d4605a42efa03229b3aa2, State: Running, Role: LEADER
I20260811 05:12:40.594269 29394 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec213fae784d4605a42efa03229b3aa2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:12:40.594259 29405 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec213fae784d4605a42efa03229b3aa2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec213fae784d4605a42efa03229b3aa2" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 45509 } }
I20260811 05:12:40.594606 29405 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec213fae784d4605a42efa03229b3aa2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec213fae784d4605a42efa03229b3aa2. Latest consensus state: current_term: 1 leader_uuid: "ec213fae784d4605a42efa03229b3aa2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec213fae784d4605a42efa03229b3aa2" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 45509 } } }
I20260811 05:12:40.594676 29405 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec213fae784d4605a42efa03229b3aa2 [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:40.595114 29408 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec213fae784d4605a42efa03229b3aa2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec213fae784d4605a42efa03229b3aa2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec213fae784d4605a42efa03229b3aa2" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 45509 } } }
I20260811 05:12:40.595217 29408 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec213fae784d4605a42efa03229b3aa2 [sys.catalog]: This master's current role is: LEADER
W20260811 05:12:40.596140 29435 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ec213fae784d4605a42efa03229b3aa2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:12:40.596200 29435 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:12:40.596249 29413 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:12:40.596412 29413 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:12:40.598042 29413 catalog_manager.cc:1347] Generated new cluster ID: 3e7bb4e216dd4843ad3c780880976b12
I20260811 05:12:40.598104 29413 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 05:12:40 dist-test-slave-f7th krb5kdc[29258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:12:40.610634 29403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:40.592176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51378 (local address 127.28.143.190:45509)
0811 05:12:40.592382 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:40.592394 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:40.592646 (+ 252us) server_negotiation.cc:407] Connection header received
0811 05:12:40.594681 (+ 2035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:40.594689 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:40.594751 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:40.594889 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:40.596869 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:40.598065 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:40.599579 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:40.600225 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:40.600359 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:40.605439 (+ 5080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:40.605445 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:40.605445 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:40.605461 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:40.607922 (+ 2461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:40.608695 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:40.608700 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:40.608700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:40.608911 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:40.609247 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:40.609248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:40.609249 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:40.609609 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:40.609885 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:40.610111 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 05:12:40.610274 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":99,"threads_started":1}
I20260811 05:12:40.624181 29413 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:12:40.625054 29413 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:12:40.636673 29413 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec213fae784d4605a42efa03229b3aa2: Generated new TSK 0
I20260811 05:12:40.636925 29413 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.143.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab.
Entry for principal kudu/127.28.143.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab.
I20260811 05:12:40.675396 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.129:0
--local_ip_for_outbound_sockets=127.28.143.129
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:45509
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:46113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:40.798172 29446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:40.798434 29446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:40.798462 29446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:40.799872 29446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:40.799934 29446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:40.799966 29446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.129
I20260811 05:12:40.802009 29446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:46113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:45509
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29446
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.129
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:12:40.802306 29446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:40.802570 29446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:40.810330 29459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:40.810417 29457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:40.811005 29456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:40.811143 29446 server_base.cc:1034] running on GCE node
I20260811 05:12:40.811398 29446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:40.811710 29446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:40.812868 29446 hybrid_clock.cc:648] HybridClock initialized: now 1786425160812834 us; error 62 us; skew 500 ppm
Aug 11 05:12:40 dist-test-slave-f7th krb5kdc[29258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425160, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:40.816394 29446 init.cc:377] Logged in from keytab as kudu/127.28.143.129@KRBTEST.COM (short username kudu)
I20260811 05:12:40.816991 29446 webserver.cc:460] Webserver started at http://127.28.143.129:38411/ using document root <none> and password file <none>
I20260811 05:12:40.817251 29446 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:40.817307 29446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:40.817442 29446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:40.818804 29446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-0/data/instance:
uuid: "710faca4f0984dda91bd806e272156a3"
format_stamp: "Formatted at 2026-08-11 05:12:40 on dist-test-slave-f7th"
server_key: "65db83e77ed70850bc06eb905c8f3b7c"
server_key_iv: "981dd96fc8aeebd79e8dcf0b7ac7a1a1"
server_key_version: "encryptionkey@0"
I20260811 05:12:40.819254 29446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance:
uuid: "710faca4f0984dda91bd806e272156a3"
format_stamp: "Formatted at 2026-08-11 05:12:40 on dist-test-slave-f7th"
server_key: "65db83e77ed70850bc06eb905c8f3b7c"
server_key_iv: "981dd96fc8aeebd79e8dcf0b7ac7a1a1"
server_key_version: "encryptionkey@0"
I20260811 05:12:40.821002 29446 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260811 05:12:40.822244 29470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:40.822469 29446 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:12:40.822605 29446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-0/wal
uuid: "710faca4f0984dda91bd806e272156a3"
format_stamp: "Formatted at 2026-08-11 05:12:40 on dist-test-slave-f7th"
server_key: "65db83e77ed70850bc06eb905c8f3b7c"
server_key_iv: "981dd96fc8aeebd79e8dcf0b7ac7a1a1"
server_key_version: "encryptionkey@0"
I20260811 05:12:40.822681 29446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:40.838762 29446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:40.841922 29446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:40.842188 29446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:40.842674 29446 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:12:40.843205 29446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:12:40.843253 29446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:40.843286 29446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:12:40.843302 29446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:40.855907 29446 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.129:45817
I20260811 05:12:40.856006 29650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.129:45817 every 8 connection(s)
I20260811 05:12:40.856473 29446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
I20260811 05:12:40.863121 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 29446
I20260811 05:12:40.863262 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance
I20260811 05:12:40.863421 29246 external_mini_cluster.cc:1428] Setting key 4ff1a9cd54fd227a962cc1ba76a51156
Aug 11 05:12:40 dist-test-slave-f7th krb5kdc[29258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425160, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:12:40.871495 29403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:40.857555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:42663 (local address 127.28.143.190:45509)
0811 05:12:40.857627 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:40.857628 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:40.857980 (+ 352us) server_negotiation.cc:407] Connection header received
0811 05:12:40.859022 (+ 1042us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:40.859023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:40.859041 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:40.859081 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:40.860348 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:40.861184 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:40.862727 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:40.863059 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:40.863141 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:40.867855 (+ 4714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:40.867858 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:40.867859 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:40.867880 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:40.869826 (+ 1946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:40.870345 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:40.870346 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:40.870347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:40.870415 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:40.870712 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:40.870713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:40.870714 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:40.870860 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:40.870926 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:40.871273 (+ 347us) server_negotiation.cc:299] Negotiation successful
0811 05:12:40.871392 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 05:12:40.871701 29659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:40.857476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:45509 (local address 127.28.143.129:42663)
0811 05:12:40.857665 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:40.857714 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:40.858828 (+ 1114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:40.859208 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:40.859212 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:40.859870 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:40.860101 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:40.860106 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:40.861389 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:40.861391 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:40.862505 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:40.862510 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:40.863255 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:40.863257 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:40.863582 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:40.864347 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:12:40.864361 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:40.867699 (+ 3338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:40.869944 (+ 2245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:40.869958 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:40.869962 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:40.870257 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:40.870484 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:40.870485 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:40.870485 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:40.870645 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:40.870926 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:40.870935 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:40.871120 (+ 185us) client_negotiation.cc:769] Sending connection context
0811 05:12:40.871230 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 05:12:40.871436 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":85,"threads_started":1}
I20260811 05:12:40.872355 29654 heartbeater.cc:347] Connected to a master server at 127.28.143.190:45509
I20260811 05:12:40.872515 29654 heartbeater.cc:464] Registering TS with master...
I20260811 05:12:40.872839 29654 heartbeater.cc:511] Master 127.28.143.190:45509 requested a full tablet report, sending...
I20260811 05:12:40.873840 29332 ts_manager.cc:194] Registered new tserver with Master: 710faca4f0984dda91bd806e272156a3 (127.28.143.129:45817)
WARNING: no policy specified for kudu/127.28.143.130@KRBTEST.COM; defaulting to no policy
I20260811 05:12:40.875087 29332 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.129@KRBTEST.COM'} at 127.28.143.129:42663
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab.
Entry for principal kudu/127.28.143.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab.
I20260811 05:12:40.904181 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.130:0
--local_ip_for_outbound_sockets=127.28.143.130
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:45509
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:46113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:40.995172 29663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:40.995520 29663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:40.995571 29663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:40.997025 29663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:40.997083 29663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:40.997128 29663 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.130
I20260811 05:12:40.999011 29663 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:46113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:45509
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29663
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.130
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:12:40.999331 29663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:40.999600 29663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:41.005699 29672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:41.005694 29675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:41.005728 29671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:41.006412 29663 server_base.cc:1034] running on GCE node
I20260811 05:12:41.006670 29663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:41.006987 29663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:41.008162 29663 hybrid_clock.cc:648] HybridClock initialized: now 1786425161008144 us; error 34 us; skew 500 ppm
Aug 11 05:12:41 dist-test-slave-f7th krb5kdc[29258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425161, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:41.011015 29663 init.cc:377] Logged in from keytab as kudu/127.28.143.130@KRBTEST.COM (short username kudu)
I20260811 05:12:41.011544 29663 webserver.cc:460] Webserver started at http://127.28.143.130:43907/ using document root <none> and password file <none>
I20260811 05:12:41.011765 29663 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:41.011804 29663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:41.011919 29663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:41.013003 29663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-1/data/instance:
uuid: "eccdd62b02dc4bafa5b7c4ffc4742458"
format_stamp: "Formatted at 2026-08-11 05:12:41 on dist-test-slave-f7th"
server_key: "445f5c49e8a59c0b7398844f8694a037"
server_key_iv: "366753e10daf40484b8e32d4c98e78e2"
server_key_version: "encryptionkey@0"
I20260811 05:12:41.013319 29663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance:
uuid: "eccdd62b02dc4bafa5b7c4ffc4742458"
format_stamp: "Formatted at 2026-08-11 05:12:41 on dist-test-slave-f7th"
server_key: "445f5c49e8a59c0b7398844f8694a037"
server_key_iv: "366753e10daf40484b8e32d4c98e78e2"
server_key_version: "encryptionkey@0"
I20260811 05:12:41.014653 29663 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 05:12:41.015453 29692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:41.015642 29663 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:12:41.015743 29663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-1/wal
uuid: "eccdd62b02dc4bafa5b7c4ffc4742458"
format_stamp: "Formatted at 2026-08-11 05:12:41 on dist-test-slave-f7th"
server_key: "445f5c49e8a59c0b7398844f8694a037"
server_key_iv: "366753e10daf40484b8e32d4c98e78e2"
server_key_version: "encryptionkey@0"
I20260811 05:12:41.015816 29663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:41.032861 29663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:41.035198 29663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:41.035329 29663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:41.035548 29663 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:12:41.036031 29663 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:12:41.036113 29663 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:41.036150 29663 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:12:41.036168 29663 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:41.042280 29663 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.130:44469
I20260811 05:12:41.042335 29873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.130:44469 every 8 connection(s)
I20260811 05:12:41.042671 29663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
Aug 11 05:12:41 dist-test-slave-f7th krb5kdc[29258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425161, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:12:41.052100 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 29663
I20260811 05:12:41.052214 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance
I20260811 05:12:41.052408 29246 external_mini_cluster.cc:1428] Setting key 6e757663c28fb62159b2ae65acbe8a1d
I20260811 05:12:41.053288 29403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:41.043522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:41807 (local address 127.28.143.190:45509)
0811 05:12:41.043625 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:41.043628 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:41.043801 (+ 173us) server_negotiation.cc:407] Connection header received
0811 05:12:41.044358 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:41.044361 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:41.044374 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:41.044435 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:41.045499 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:41.046028 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:41.047137 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:41.047339 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:41.047407 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:41.049892 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:41.049895 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:41.049895 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:41.049904 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:41.051427 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:41.051900 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:41.051902 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:41.051902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:41.052007 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:41.052376 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:41.052378 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:41.052379 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:41.052544 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:41.052654 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:41.053094 (+ 440us) server_negotiation.cc:299] Negotiation successful
0811 05:12:41.053197 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 05:12:41.053434 29878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:41.043448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:45509 (local address 127.28.143.130:41807)
0811 05:12:41.043664 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:41.043707 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:41.044248 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:41.044511 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:41.044514 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:41.045152 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:41.045374 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:41.045379 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:41.046129 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:41.046131 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:41.047037 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:41.047041 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:41.047415 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:41.047417 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:41.047556 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:41.048185 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:12:41.048195 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:41.049779 (+ 1584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:41.051534 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:41.051542 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:41.051544 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:41.051805 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:41.052092 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:41.052093 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:41.052094 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:41.052256 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:41.052654 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:41.052662 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:41.052812 (+ 150us) client_negotiation.cc:769] Sending connection context
0811 05:12:41.053022 (+ 210us) client_negotiation.cc:240] Negotiation successful
0811 05:12:41.053221 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":96,"threads_started":1}
I20260811 05:12:41.053946 29875 heartbeater.cc:347] Connected to a master server at 127.28.143.190:45509
I20260811 05:12:41.054068 29875 heartbeater.cc:464] Registering TS with master...
I20260811 05:12:41.054271 29875 heartbeater.cc:511] Master 127.28.143.190:45509 requested a full tablet report, sending...
I20260811 05:12:41.054636 29332 ts_manager.cc:194] Registered new tserver with Master: eccdd62b02dc4bafa5b7c4ffc4742458 (127.28.143.130:44469)
I20260811 05:12:41.055033 29332 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.130@KRBTEST.COM'} at 127.28.143.130:41807
WARNING: no policy specified for kudu/127.28.143.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab.
Entry for principal kudu/127.28.143.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab.
I20260811 05:12:41.084668 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.131:0
--local_ip_for_outbound_sockets=127.28.143.131
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:45509
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:46113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:41.170025 29882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:41.170260 29882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:41.170286 29882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:41.171556 29882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:41.171607 29882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:41.171636 29882 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.131
I20260811 05:12:41.173198 29882 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:46113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:45509
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29882
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.131
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:12:41.173440 29882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:41.173671 29882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:41.178851 29899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:41.178895 29895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:41.179102 29882 server_base.cc:1034] running on GCE node
W20260811 05:12:41.179131 29891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:41.179385 29882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:41.179682 29882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:41.180873 29882 hybrid_clock.cc:648] HybridClock initialized: now 1786425161180817 us; error 68 us; skew 500 ppm
Aug 11 05:12:41 dist-test-slave-f7th krb5kdc[29258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425161, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:41.183789 29882 init.cc:377] Logged in from keytab as kudu/127.28.143.131@KRBTEST.COM (short username kudu)
I20260811 05:12:41.184350 29882 webserver.cc:460] Webserver started at http://127.28.143.131:35799/ using document root <none> and password file <none>
I20260811 05:12:41.184645 29882 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:41.184716 29882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:41.184859 29882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:41.185954 29882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-2/data/instance:
uuid: "502eeed38e634a5ab971459e5c1c088b"
format_stamp: "Formatted at 2026-08-11 05:12:41 on dist-test-slave-f7th"
server_key: "30f3627c30b89f5e863e9e26bb92dea1"
server_key_iv: "0c6e334991a11c85f40b81fafb0098c0"
server_key_version: "encryptionkey@0"
I20260811 05:12:41.186303 29882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance:
uuid: "502eeed38e634a5ab971459e5c1c088b"
format_stamp: "Formatted at 2026-08-11 05:12:41 on dist-test-slave-f7th"
server_key: "30f3627c30b89f5e863e9e26bb92dea1"
server_key_iv: "0c6e334991a11c85f40b81fafb0098c0"
server_key_version: "encryptionkey@0"
I20260811 05:12:41.187772 29882 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 05:12:41.188632 29910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:41.188824 29882 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:12:41.188923 29882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-2/wal
uuid: "502eeed38e634a5ab971459e5c1c088b"
format_stamp: "Formatted at 2026-08-11 05:12:41 on dist-test-slave-f7th"
server_key: "30f3627c30b89f5e863e9e26bb92dea1"
server_key_iv: "0c6e334991a11c85f40b81fafb0098c0"
server_key_version: "encryptionkey@0"
I20260811 05:12:41.188999 29882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:41.197032 29882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:41.199432 29882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:41.199599 29882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:41.199859 29882 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:12:41.200196 29882 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:12:41.200232 29882 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:41.200263 29882 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:12:41.200287 29882 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:41.207888 29882 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.131:43457
I20260811 05:12:41.208010 30110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.131:43457 every 8 connection(s)
I20260811 05:12:41.208320 29882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
I20260811 05:12:41.210686 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 29882
I20260811 05:12:41.210760 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance
I20260811 05:12:41.210889 29246 external_mini_cluster.cc:1428] Setting key 1ad948561a92b574ac14b40c91b8f48b
Aug 11 05:12:41 dist-test-slave-f7th krb5kdc[29258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425161, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:12:41.219445 29403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:41.209621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:48083 (local address 127.28.143.190:45509)
0811 05:12:41.209742 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:41.209821 (+ 79us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:41.209836 (+ 15us) server_negotiation.cc:407] Connection header received
0811 05:12:41.210332 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:41.210334 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:41.210349 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:41.210415 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:41.211546 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:41.212249 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:41.213313 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:41.213524 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:41.213596 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:41.216551 (+ 2955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:41.216553 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:41.216554 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:41.216563 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:41.217939 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:41.218405 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:41.218405 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:41.218406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:41.218460 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:41.218771 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:41.218772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:41.218772 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:41.218893 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:41.218944 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:41.219298 (+ 354us) server_negotiation.cc:299] Negotiation successful
0811 05:12:41.219375 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 05:12:41.219676 30115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:41.209484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:45509 (local address 127.28.143.131:48083)
0811 05:12:41.209665 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:41.209711 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:41.210197 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:41.210473 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:41.210476 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:41.211151 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:41.211425 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:41.211433 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:41.212325 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:41.212326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:41.213212 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:41.213215 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:41.213729 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:41.213733 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:41.213962 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:41.214836 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:12:41.214848 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:41.216415 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:41.218040 (+ 1625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:41.218087 (+ 47us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:41.218090 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:41.218337 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:41.218541 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:41.218542 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:41.218543 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:41.218701 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:41.218997 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:41.219004 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:41.219160 (+ 156us) client_negotiation.cc:769] Sending connection context
0811 05:12:41.219263 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 05:12:41.219423 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":86,"threads_started":1}
I20260811 05:12:41.220361 30112 heartbeater.cc:347] Connected to a master server at 127.28.143.190:45509
I20260811 05:12:41.220496 30112 heartbeater.cc:464] Registering TS with master...
I20260811 05:12:41.220727 30112 heartbeater.cc:511] Master 127.28.143.190:45509 requested a full tablet report, sending...
I20260811 05:12:41.221166 29332 ts_manager.cc:194] Registered new tserver with Master: 502eeed38e634a5ab971459e5c1c088b (127.28.143.131:43457)
I20260811 05:12:41.221550 29332 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.131@KRBTEST.COM'} at 127.28.143.131:48083
I20260811 05:12:41.223133 29246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 05:12:41 dist-test-slave-f7th krb5kdc[29258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425161, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 05:12:41 dist-test-slave-f7th krb5kdc[29258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425161, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.28.143.129@KRBTEST.COM
I20260811 05:12:41.250353 30123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:41.241382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42540 (local address 127.28.143.129:45817)
0811 05:12:41.241571 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:41.241576 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:41.241587 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:12:41.241615 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:41.241617 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:41.241632 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:41.241697 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:41.242287 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:41.243078 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:41.243854 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:41.244202 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:41.244269 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:41.247092 (+ 2823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:41.247102 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:41.247102 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:41.247111 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:41.248922 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:41.249388 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:41.249393 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:41.249393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:41.249507 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:41.249724 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:41.249725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:41.249725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:41.249909 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:41.250097 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:41.250148 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 05:12:41.250243 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":99,"threads_started":1}
W20260811 05:12:41.250897 29512 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42540
Aug 11 05:12:41 dist-test-slave-f7th krb5kdc[29258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425161, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 05:12:41 dist-test-slave-f7th krb5kdc[29258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425161, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.143.129@KRBTEST.COM
I20260811 05:12:41.277896 30123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:41.269070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42546 (local address 127.28.143.129:45817)
0811 05:12:41.269137 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:41.269139 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:41.269172 (+ 33us) server_negotiation.cc:407] Connection header received
0811 05:12:41.269247 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:41.269248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:41.269258 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:41.269309 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:41.269919 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:41.270426 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:41.271201 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:41.271408 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:41.271465 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:41.274661 (+ 3196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:41.274663 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:41.274664 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:41.274673 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:41.276709 (+ 2036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:41.277110 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:41.277112 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:41.277112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:41.277173 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:41.277367 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:41.277369 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:41.277369 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:41.277485 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:41.277605 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:41.277698 (+ 93us) server_negotiation.cc:299] Negotiation successful
0811 05:12:41.277798 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 05:12:41.278558 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 29446
I20260811 05:12:41.298861 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 29663
I20260811 05:12:41.312139 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 29882
I20260811 05:12:41.333521 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 29284
2026-08-11T05:12:41Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2294 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:12:41 dist-test-slave-f7th krb5kdc[30136](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:12:41 dist-test-slave-f7th krb5kdc[30136](info): set up 2 sockets
Aug 11 05:12:41 dist-test-slave-f7th krb5kdc[30136](info): commencing operation
krb5kdc: starting...
W20260811 05:12:42.567131 29246 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Aug 11 05:12:42 dist-test-slave-f7th krb5kdc[30136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425162, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:12:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:12:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
I20260811 05:12:42.699355 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:45963
--webserver_interface=127.28.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:35411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:45963
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:42.790895 30166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:42.791152 30166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:42.791186 30166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:42.792451 30166 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:12:42.792505 30166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:42.792522 30166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:42.792538 30166 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:12:42.792554 30166 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:12:42.794252 30166 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:35411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:45963
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:45963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:12:42.794543 30166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:42.794764 30166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:42.799742 30172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:42.799695 30179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:42.799724 30166 server_base.cc:1034] running on GCE node
W20260811 05:12:42.799695 30176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:42.800215 30166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:42.800525 30166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:42.801692 30166 hybrid_clock.cc:648] HybridClock initialized: now 1786425162801685 us; error 37 us; skew 500 ppm
Aug 11 05:12:42 dist-test-slave-f7th krb5kdc[30136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425162, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:42.804327 30166 init.cc:377] Logged in from keytab as kudu/127.28.143.190@KRBTEST.COM (short username kudu)
I20260811 05:12:42.804855 30166 webserver.cc:460] Webserver started at http://127.28.143.190:35499/ using document root <none> and password file <none>
I20260811 05:12:42.805097 30166 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:42.805148 30166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:42.805277 30166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:42.806427 30166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "4005c72f6db947ef8e5a72c1383dfee3"
format_stamp: "Formatted at 2026-08-11 05:12:42 on dist-test-slave-f7th"
server_key: "c5fe5d6975c09a3eec577a4f3db5490e"
server_key_iv: "d88fae342541f6767fdc5fac0d91de3c"
server_key_version: "encryptionkey@0"
I20260811 05:12:42.806777 30166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "4005c72f6db947ef8e5a72c1383dfee3"
format_stamp: "Formatted at 2026-08-11 05:12:42 on dist-test-slave-f7th"
server_key: "c5fe5d6975c09a3eec577a4f3db5490e"
server_key_iv: "d88fae342541f6767fdc5fac0d91de3c"
server_key_version: "encryptionkey@0"
I20260811 05:12:42.808164 30166 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 05:12:42.809031 30187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:42.809276 30166 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:12:42.809389 30166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "4005c72f6db947ef8e5a72c1383dfee3"
format_stamp: "Formatted at 2026-08-11 05:12:42 on dist-test-slave-f7th"
server_key: "c5fe5d6975c09a3eec577a4f3db5490e"
server_key_iv: "d88fae342541f6767fdc5fac0d91de3c"
server_key_version: "encryptionkey@0"
I20260811 05:12:42.809456 30166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:42.822495 30166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:42.825555 30166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:42.825759 30166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:42.830170 30166 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:45963
I20260811 05:12:42.830224 30291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:45963 every 8 connection(s)
I20260811 05:12:42.830580 30166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:12:42.831512 30292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:42.833954 30292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4005c72f6db947ef8e5a72c1383dfee3: Bootstrap starting.
I20260811 05:12:42.834568 30292 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4005c72f6db947ef8e5a72c1383dfee3: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:42.834857 30292 log.cc:824] T 00000000000000000000000000000000 P 4005c72f6db947ef8e5a72c1383dfee3: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:42.835632 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 30166
I20260811 05:12:42.835669 30292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4005c72f6db947ef8e5a72c1383dfee3: No bootstrap required, opened a new log
I20260811 05:12:42.835717 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:12:42.835852 29246 external_mini_cluster.cc:1428] Setting key efd477435feab014c67d5065179f6324
I20260811 05:12:42.837739 30292 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4005c72f6db947ef8e5a72c1383dfee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4005c72f6db947ef8e5a72c1383dfee3" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 45963 } }
I20260811 05:12:42.837940 30292 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4005c72f6db947ef8e5a72c1383dfee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:42.837981 30292 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4005c72f6db947ef8e5a72c1383dfee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4005c72f6db947ef8e5a72c1383dfee3, State: Initialized, Role: FOLLOWER
I20260811 05:12:42.838143 30292 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4005c72f6db947ef8e5a72c1383dfee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4005c72f6db947ef8e5a72c1383dfee3" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 45963 } }
I20260811 05:12:42.838224 30292 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4005c72f6db947ef8e5a72c1383dfee3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:12:42.838248 30292 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4005c72f6db947ef8e5a72c1383dfee3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:12:42.838297 30292 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4005c72f6db947ef8e5a72c1383dfee3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:42.839483 30292 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4005c72f6db947ef8e5a72c1383dfee3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4005c72f6db947ef8e5a72c1383dfee3" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 45963 } }
I20260811 05:12:42.839619 30292 leader_election.cc:302] T 00000000000000000000000000000000 P 4005c72f6db947ef8e5a72c1383dfee3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4005c72f6db947ef8e5a72c1383dfee3; no voters:
I20260811 05:12:42.839797 30292 leader_election.cc:288] T 00000000000000000000000000000000 P 4005c72f6db947ef8e5a72c1383dfee3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:12:42.839915 30300 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4005c72f6db947ef8e5a72c1383dfee3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:42.840063 30292 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4005c72f6db947ef8e5a72c1383dfee3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:12:42.840070 30300 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4005c72f6db947ef8e5a72c1383dfee3 [term 1 LEADER]: Becoming Leader. State: Replica: 4005c72f6db947ef8e5a72c1383dfee3, State: Running, Role: LEADER
I20260811 05:12:42.841158 30300 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4005c72f6db947ef8e5a72c1383dfee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4005c72f6db947ef8e5a72c1383dfee3" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 45963 } }
I20260811 05:12:42.841836 30301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4005c72f6db947ef8e5a72c1383dfee3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4005c72f6db947ef8e5a72c1383dfee3. Latest consensus state: current_term: 1 leader_uuid: "4005c72f6db947ef8e5a72c1383dfee3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4005c72f6db947ef8e5a72c1383dfee3" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 45963 } } }
I20260811 05:12:42.841840 30302 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4005c72f6db947ef8e5a72c1383dfee3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4005c72f6db947ef8e5a72c1383dfee3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4005c72f6db947ef8e5a72c1383dfee3" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 45963 } } }
I20260811 05:12:42.842092 30301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4005c72f6db947ef8e5a72c1383dfee3 [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:42.842099 30302 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4005c72f6db947ef8e5a72c1383dfee3 [sys.catalog]: This master's current role is: LEADER
W20260811 05:12:42.842723 30326 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4005c72f6db947ef8e5a72c1383dfee3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:12:42.842779 30326 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:12:42.842825 30330 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:12:42.843001 30330 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 11 05:12:42 dist-test-slave-f7th krb5kdc[30136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425162, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:12:42.845039 30330 catalog_manager.cc:1347] Generated new cluster ID: 27048726d7964968b5944e4f7539c024
I20260811 05:12:42.845120 30330 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:12:42.848682 30299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:42.836422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49606 (local address 127.28.143.190:45963)
0811 05:12:42.836660 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:42.836671 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:42.836707 (+ 36us) server_negotiation.cc:407] Connection header received
0811 05:12:42.837695 (+ 988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:42.837705 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:42.837776 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:42.837938 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:42.838669 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:42.840078 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:42.841062 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:42.841655 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:42.841774 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:42.845250 (+ 3476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:42.845255 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:42.845255 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:42.845272 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:42.847032 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:42.847411 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:42.847415 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:42.847415 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:42.847546 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:42.847742 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:42.847743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:42.847744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:42.848002 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:42.848157 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:42.848273 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 05:12:42.848419 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":100,"threads_started":1}
I20260811 05:12:42.865108 30330 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:12:42.865877 30330 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:12:42.870406 30330 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4005c72f6db947ef8e5a72c1383dfee3: Generated new TSK 0
I20260811 05:12:42.870646 30330 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.143.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab.
Entry for principal kudu/127.28.143.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab.
I20260811 05:12:42.912745 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.129:0
--local_ip_for_outbound_sockets=127.28.143.129
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:45963
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:35411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:43.030117 30339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:43.030361 30339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:43.030398 30339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:43.031657 30339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:43.031708 30339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:43.031745 30339 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.129
I20260811 05:12:43.033360 30339 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:35411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:45963
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30339
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.129
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:12:43.033638 30339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:43.033857 30339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:43.039191 30357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:43.039176 30339 server_base.cc:1034] running on GCE node
W20260811 05:12:43.039216 30354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:43.039176 30355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:43.039652 30339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:43.039954 30339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:43.041114 30339 hybrid_clock.cc:648] HybridClock initialized: now 1786425163041096 us; error 36 us; skew 500 ppm
Aug 11 05:12:43 dist-test-slave-f7th krb5kdc[30136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425163, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:43.043929 30339 init.cc:377] Logged in from keytab as kudu/127.28.143.129@KRBTEST.COM (short username kudu)
I20260811 05:12:43.044395 30339 webserver.cc:460] Webserver started at http://127.28.143.129:33129/ using document root <none> and password file <none>
I20260811 05:12:43.044608 30339 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:43.044694 30339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:43.044814 30339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:43.045913 30339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-0/data/instance:
uuid: "6a3f42398e9f43acaf35f9e4c953235a"
format_stamp: "Formatted at 2026-08-11 05:12:43 on dist-test-slave-f7th"
server_key: "c42585eeb034451b03c8647db0a2f427"
server_key_iv: "ba16313cad2fa730d6e7653d5cd34f41"
server_key_version: "encryptionkey@0"
I20260811 05:12:43.046252 30339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance:
uuid: "6a3f42398e9f43acaf35f9e4c953235a"
format_stamp: "Formatted at 2026-08-11 05:12:43 on dist-test-slave-f7th"
server_key: "c42585eeb034451b03c8647db0a2f427"
server_key_iv: "ba16313cad2fa730d6e7653d5cd34f41"
server_key_version: "encryptionkey@0"
I20260811 05:12:43.047688 30339 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 05:12:43.048609 30371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:43.048826 30339 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:12:43.048921 30339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-0/wal
uuid: "6a3f42398e9f43acaf35f9e4c953235a"
format_stamp: "Formatted at 2026-08-11 05:12:43 on dist-test-slave-f7th"
server_key: "c42585eeb034451b03c8647db0a2f427"
server_key_iv: "ba16313cad2fa730d6e7653d5cd34f41"
server_key_version: "encryptionkey@0"
I20260811 05:12:43.048985 30339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:43.063556 30339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:43.066210 30339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:43.066340 30339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:43.066569 30339 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:12:43.066883 30339 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:12:43.066916 30339 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:43.066938 30339 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:12:43.066957 30339 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:43.072888 30339 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.129:45063
I20260811 05:12:43.072922 30577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.129:45063 every 8 connection(s)
I20260811 05:12:43.073282 30339 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
Aug 11 05:12:43 dist-test-slave-f7th krb5kdc[30136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425163, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:12:43.080960 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 30339
I20260811 05:12:43.081061 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance
I20260811 05:12:43.081228 29246 external_mini_cluster.cc:1428] Setting key ee0fafc49a1e6f3129e24e579a88de0d
I20260811 05:12:43.083372 30299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:43.074097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:44723 (local address 127.28.143.190:45963)
0811 05:12:43.074175 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:43.074178 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:43.074405 (+ 227us) server_negotiation.cc:407] Connection header received
0811 05:12:43.075399 (+ 994us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:43.075400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:43.075413 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:43.075468 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:43.076180 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:43.076740 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:43.077746 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:43.077954 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:43.078012 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:43.080272 (+ 2260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:43.080274 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:43.080275 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:43.080283 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:43.081857 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:43.082300 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:43.082302 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:43.082302 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:43.082374 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:43.082614 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:43.082615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:43.082615 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:43.082750 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:43.082816 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:43.083158 (+ 342us) server_negotiation.cc:299] Negotiation successful
0811 05:12:43.083267 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 05:12:43.083456 30581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:43.074060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:45963 (local address 127.28.143.129:44723)
0811 05:12:43.074240 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:43.074279 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:43.075277 (+ 998us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:43.075552 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:43.075556 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:43.075839 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:43.076078 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:43.076082 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:43.076831 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:43.076832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:43.077663 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:43.077665 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:43.078026 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:43.078027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:43.078143 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:43.078657 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:12:43.078667 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:43.080170 (+ 1503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:43.081949 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:43.081959 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:43.081962 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:43.082225 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:43.082445 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:43.082446 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:43.082446 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:43.082558 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:43.082837 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:43.082845 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:43.082987 (+ 142us) client_negotiation.cc:769] Sending connection context
0811 05:12:43.083121 (+ 134us) client_negotiation.cc:240] Negotiation successful
0811 05:12:43.083270 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":80,"threads_started":1}
I20260811 05:12:43.084127 30579 heartbeater.cc:347] Connected to a master server at 127.28.143.190:45963
I20260811 05:12:43.084231 30579 heartbeater.cc:464] Registering TS with master...
I20260811 05:12:43.084420 30579 heartbeater.cc:511] Master 127.28.143.190:45963 requested a full tablet report, sending...
I20260811 05:12:43.085048 30219 ts_manager.cc:194] Registered new tserver with Master: 6a3f42398e9f43acaf35f9e4c953235a (127.28.143.129:45063)
I20260811 05:12:43.085745 30219 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.129@KRBTEST.COM'} at 127.28.143.129:44723
WARNING: no policy specified for kudu/127.28.143.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab.
Entry for principal kudu/127.28.143.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab.
I20260811 05:12:43.111130 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.130:0
--local_ip_for_outbound_sockets=127.28.143.130
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:45963
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:35411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:43.197073 30589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:43.197281 30589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:43.197306 30589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:43.198493 30589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:43.198540 30589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:43.198567 30589 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.130
I20260811 05:12:43.200132 30589 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:35411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:45963
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.130
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:12:43.200393 30589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:43.200651 30589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:43.205924 30599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:43.205916 30596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:43.205914 30604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:43.206298 30589 server_base.cc:1034] running on GCE node
I20260811 05:12:43.206636 30589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:43.207012 30589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:43.208194 30589 hybrid_clock.cc:648] HybridClock initialized: now 1786425163208162 us; error 52 us; skew 500 ppm
Aug 11 05:12:43 dist-test-slave-f7th krb5kdc[30136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425163, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:43.211182 30589 init.cc:377] Logged in from keytab as kudu/127.28.143.130@KRBTEST.COM (short username kudu)
I20260811 05:12:43.211679 30589 webserver.cc:460] Webserver started at http://127.28.143.130:37581/ using document root <none> and password file <none>
I20260811 05:12:43.211935 30589 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:43.212100 30589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:43.212323 30589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:43.213549 30589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-1/data/instance:
uuid: "e9e36a61a9bc4e51ba87790b93fe9742"
format_stamp: "Formatted at 2026-08-11 05:12:43 on dist-test-slave-f7th"
server_key: "23457bbba18108194bd81571529ab15a"
server_key_iv: "dc4d52b431fdec139012d144667f81b0"
server_key_version: "encryptionkey@0"
I20260811 05:12:43.213908 30589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance:
uuid: "e9e36a61a9bc4e51ba87790b93fe9742"
format_stamp: "Formatted at 2026-08-11 05:12:43 on dist-test-slave-f7th"
server_key: "23457bbba18108194bd81571529ab15a"
server_key_iv: "dc4d52b431fdec139012d144667f81b0"
server_key_version: "encryptionkey@0"
I20260811 05:12:43.215394 30589 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 05:12:43.216449 30613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:43.216714 30589 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 05:12:43.216820 30589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-1/wal
uuid: "e9e36a61a9bc4e51ba87790b93fe9742"
format_stamp: "Formatted at 2026-08-11 05:12:43 on dist-test-slave-f7th"
server_key: "23457bbba18108194bd81571529ab15a"
server_key_iv: "dc4d52b431fdec139012d144667f81b0"
server_key_version: "encryptionkey@0"
I20260811 05:12:43.216912 30589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:43.233292 30589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:43.235767 30589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:43.236073 30589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:43.236361 30589 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:12:43.236754 30589 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:12:43.236800 30589 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:43.236824 30589 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:12:43.236836 30589 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:43.242846 30589 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.130:34117
I20260811 05:12:43.242909 30807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.130:34117 every 8 connection(s)
I20260811 05:12:43.243242 30589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
I20260811 05:12:43.247225 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 30589
I20260811 05:12:43.247334 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance
I20260811 05:12:43.247524 29246 external_mini_cluster.cc:1428] Setting key 096f51918bab223361f23f5b78b09b70
Aug 11 05:12:43 dist-test-slave-f7th krb5kdc[30136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425163, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:12:43.257938 30299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:43.244055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:46707 (local address 127.28.143.190:45963)
0811 05:12:43.244138 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:43.244141 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:43.244325 (+ 184us) server_negotiation.cc:407] Connection header received
0811 05:12:43.245367 (+ 1042us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:43.245369 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:43.245382 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:43.245443 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:43.246186 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:43.247080 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:43.248652 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:43.248909 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:43.248979 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:43.252209 (+ 3230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:43.252211 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:43.252211 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:43.252222 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:43.256014 (+ 3792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:43.256597 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:43.256599 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:43.256599 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:43.256698 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:43.257125 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:43.257126 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:43.257126 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:43.257291 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:43.257368 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:43.257725 (+ 357us) server_negotiation.cc:299] Negotiation successful
0811 05:12:43.257838 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
WARNING: no policy specified for kudu/127.28.143.131@KRBTEST.COM; defaulting to no policy
I20260811 05:12:43.258059 30821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:43.244017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:45963 (local address 127.28.143.130:46707)
0811 05:12:43.244195 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:43.244235 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:43.245224 (+ 989us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:43.245515 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:43.245519 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:43.245847 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:43.246065 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:43.246070 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:43.247220 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:43.247221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:43.248483 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:43.248487 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:43.249023 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:43.249025 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:43.249282 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:43.250060 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:12:43.250073 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:43.251981 (+ 1908us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:43.256167 (+ 4186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:43.256178 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:43.256180 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:43.256496 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:43.256814 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:43.256815 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:43.256815 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:43.257038 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:43.257402 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:43.257410 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:43.257573 (+ 163us) client_negotiation.cc:769] Sending connection context
0811 05:12:43.257673 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 05:12:43.257834 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":84,"threads_started":1}
I20260811 05:12:43.258703 30816 heartbeater.cc:347] Connected to a master server at 127.28.143.190:45963
I20260811 05:12:43.258816 30816 heartbeater.cc:464] Registering TS with master...
I20260811 05:12:43.259068 30816 heartbeater.cc:511] Master 127.28.143.190:45963 requested a full tablet report, sending...
I20260811 05:12:43.259521 30219 ts_manager.cc:194] Registered new tserver with Master: e9e36a61a9bc4e51ba87790b93fe9742 (127.28.143.130:34117)
I20260811 05:12:43.259971 30219 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.130@KRBTEST.COM'} at 127.28.143.130:46707
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab.
Entry for principal kudu/127.28.143.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab.
I20260811 05:12:43.285382 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.131:0
--local_ip_for_outbound_sockets=127.28.143.131
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:45963
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:35411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:43.372267 30833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:43.372491 30833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:43.372519 30833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:43.373840 30833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:43.373903 30833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:43.373934 30833 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.131
I20260811 05:12:43.375579 30833 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:35411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:45963
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30833
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.131
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:12:43.375821 30833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:43.376046 30833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:43.381430 30841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:43.381440 30846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:43.381430 30838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:43.382202 30833 server_base.cc:1034] running on GCE node
I20260811 05:12:43.382421 30833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:43.382683 30833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:43.383890 30833 hybrid_clock.cc:648] HybridClock initialized: now 1786425163383867 us; error 37 us; skew 500 ppm
Aug 11 05:12:43 dist-test-slave-f7th krb5kdc[30136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425163, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:43.386822 30833 init.cc:377] Logged in from keytab as kudu/127.28.143.131@KRBTEST.COM (short username kudu)
I20260811 05:12:43.387476 30833 webserver.cc:460] Webserver started at http://127.28.143.131:43111/ using document root <none> and password file <none>
I20260811 05:12:43.387728 30833 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:43.387781 30833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:43.387931 30833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:43.389315 30833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-2/data/instance:
uuid: "7aee251e414d44d98fad9f6bc76a7b9d"
format_stamp: "Formatted at 2026-08-11 05:12:43 on dist-test-slave-f7th"
server_key: "d61f2dfd1e567e62b30d869a9f5d3ebf"
server_key_iv: "93baf86a2474e9db69d52e8e6950d92f"
server_key_version: "encryptionkey@0"
I20260811 05:12:43.389935 30833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance:
uuid: "7aee251e414d44d98fad9f6bc76a7b9d"
format_stamp: "Formatted at 2026-08-11 05:12:43 on dist-test-slave-f7th"
server_key: "d61f2dfd1e567e62b30d869a9f5d3ebf"
server_key_iv: "93baf86a2474e9db69d52e8e6950d92f"
server_key_version: "encryptionkey@0"
I20260811 05:12:43.391479 30833 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 05:12:43.392481 30860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:43.392786 30833 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:12:43.392892 30833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-2/wal
uuid: "7aee251e414d44d98fad9f6bc76a7b9d"
format_stamp: "Formatted at 2026-08-11 05:12:43 on dist-test-slave-f7th"
server_key: "d61f2dfd1e567e62b30d869a9f5d3ebf"
server_key_iv: "93baf86a2474e9db69d52e8e6950d92f"
server_key_version: "encryptionkey@0"
I20260811 05:12:43.392963 30833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:43.418835 30833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:43.421926 30833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:43.422151 30833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:43.422446 30833 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:12:43.422803 30833 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:12:43.422840 30833 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:43.422873 30833 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:12:43.422889 30833 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:43.429049 30833 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.131:46285
I20260811 05:12:43.429121 31076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.131:46285 every 8 connection(s)
I20260811 05:12:43.429469 30833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
I20260811 05:12:43.432987 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 30833
I20260811 05:12:43.433183 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance
I20260811 05:12:43.433410 29246 external_mini_cluster.cc:1428] Setting key fc3507d7347c54489927acb0b5771495
Aug 11 05:12:43 dist-test-slave-f7th krb5kdc[30136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425163, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:12:43.442142 30299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:43.430377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:57061 (local address 127.28.143.190:45963)
0811 05:12:43.430471 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:43.430474 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:43.430654 (+ 180us) server_negotiation.cc:407] Connection header received
0811 05:12:43.431779 (+ 1125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:43.431781 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:43.431794 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:43.431914 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:43.432833 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:43.433663 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:43.435353 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:43.435651 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:43.435719 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:43.438459 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:43.438461 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:43.438462 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:43.438472 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:43.440429 (+ 1957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:43.440994 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:43.440995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:43.440996 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:43.441062 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:43.441381 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:43.441382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:43.441382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:43.441529 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:43.441594 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:43.441920 (+ 326us) server_negotiation.cc:299] Negotiation successful
0811 05:12:43.442024 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 05:12:43.442349 31080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:43.430308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:45963 (local address 127.28.143.131:57061)
0811 05:12:43.430487 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:43.430532 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:43.431626 (+ 1094us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:43.432005 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:43.432010 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:43.432313 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:43.432528 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:43.432532 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:43.433836 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:43.433838 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:43.435228 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:43.435233 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:43.435769 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:43.435771 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:43.435917 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:43.436543 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:12:43.436554 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:43.438312 (+ 1758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:43.440539 (+ 2227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:43.440553 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:43.440557 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:43.440909 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:43.441135 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:43.441136 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:43.441137 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:43.441318 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:43.441602 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:43.441611 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:43.441782 (+ 171us) client_negotiation.cc:769] Sending connection context
0811 05:12:43.441887 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 05:12:43.442089 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":99,"threads_started":1}
I20260811 05:12:43.442939 31077 heartbeater.cc:347] Connected to a master server at 127.28.143.190:45963
I20260811 05:12:43.443107 31077 heartbeater.cc:464] Registering TS with master...
I20260811 05:12:43.443336 31077 heartbeater.cc:511] Master 127.28.143.190:45963 requested a full tablet report, sending...
I20260811 05:12:43.443796 30219 ts_manager.cc:194] Registered new tserver with Master: 7aee251e414d44d98fad9f6bc76a7b9d (127.28.143.131:46285)
I20260811 05:12:43.444211 30219 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.131@KRBTEST.COM'} at 127.28.143.131:57061
I20260811 05:12:43.446381 29246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:12:43.454712 30299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:43.448280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49614 (local address 127.28.143.190:45963)
0811 05:12:43.448345 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:43.448347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:43.448445 (+ 98us) server_negotiation.cc:407] Connection header received
0811 05:12:43.448476 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:43.448477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:43.448489 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:43.448548 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:43.449193 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:43.449708 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:43.450721 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:43.450924 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:43.451059 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:43.452336 (+ 1277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:43.452337 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:43.452338 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:43.452345 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:43.453727 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:43.454081 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:43.454082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:43.454082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:43.454136 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:43.454270 (+ 134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:43.454270 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:43.454270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:43.454420 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:43.454499 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:43.454572 (+ 73us) server_negotiation.cc:299] Negotiation successful
0811 05:12:43.454638 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 05:12:43.457911 30219 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49614:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:12:43.459626 30219 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:12:43.467602 31094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:43.462581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58878 (local address 127.28.143.131:46285)
0811 05:12:43.462813 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:43.462818 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:43.462946 (+ 128us) server_negotiation.cc:407] Connection header received
0811 05:12:43.463100 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:43.463103 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:43.463122 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:43.463202 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:43.463344 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:43.464214 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:43.466203 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:43.467110 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:43.467186 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:43.467256 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:43.467451 (+ 195us) server_negotiation.cc:299] Negotiation successful
0811 05:12:43.467500 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":100,"threads_started":1}
I20260811 05:12:43.467628 31096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:43.462473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.131:46285 (local address 127.0.0.1:58878)
0811 05:12:43.462872 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:43.462884 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:43.462975 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:43.463199 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:43.463200 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:43.463202 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:43.463247 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:43.463250 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:43.464320 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:43.464321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:43.466096 (+ 1775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:43.466100 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:43.467211 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:43.467213 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:43.467313 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:43.467331 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 05:12:43.467427 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 05:12:43.467495 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"mutex_wait_us":34,"thread_start_us":102,"threads_started":1}
I20260811 05:12:43.468400 31092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:43.462700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:34117 (local address 127.0.0.1:60820)
0811 05:12:43.462921 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:43.462929 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:43.462975 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:43.463361 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:43.463362 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:43.463365 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:43.463410 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:43.463412 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:43.465896 (+ 2484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:43.465898 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:43.467057 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:43.467061 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:43.468132 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:43.468133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:43.468200 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:43.468205 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 05:12:43.468271 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 05:12:43.468320 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":78,"threads_started":1}
I20260811 05:12:43.468408 31098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:43.462954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60820 (local address 127.28.143.130:34117)
0811 05:12:43.463135 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:43.463139 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:43.463151 (+ 12us) server_negotiation.cc:407] Connection header received
0811 05:12:43.463186 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:43.463188 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:43.463203 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:43.463270 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:43.463488 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:43.464552 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:43.467204 (+ 2652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:43.468041 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:43.468118 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:43.468174 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:43.468289 (+ 115us) server_negotiation.cc:299] Negotiation successful
0811 05:12:43.468332 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":130,"threads_started":1}
I20260811 05:12:43.468571 30942 tablet_service.cc:1467] Processing CreateTablet for tablet bab39b63a940426bb5d312f7ea9d8947 (DEFAULT_TABLE table=test-table [id=8b231fd83dec47afa932df9b5b62b199]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:12:43.469017 31093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:43.462463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:45063 (local address 127.0.0.1:60136)
0811 05:12:43.462633 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:43.462681 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:43.462755 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:43.463155 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:43.463158 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:43.463161 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:43.463249 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:43.463250 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:43.464736 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:43.464737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:43.466097 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:43.466100 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:43.468403 (+ 2303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:43.468404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:43.468769 (+ 365us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:43.468775 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 05:12:43.468902 (+ 127us) client_negotiation.cc:240] Negotiation successful
0811 05:12:43.468950 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":7,"thread_start_us":73,"threads_started":1}
I20260811 05:12:43.469054 30942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bab39b63a940426bb5d312f7ea9d8947. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:43.469201 30697 tablet_service.cc:1467] Processing CreateTablet for tablet bab39b63a940426bb5d312f7ea9d8947 (DEFAULT_TABLE table=test-table [id=8b231fd83dec47afa932df9b5b62b199]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:12:43.469383 31097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:43.462684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60136 (local address 127.28.143.129:45063)
0811 05:12:43.462894 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:43.462898 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:43.462909 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:12:43.462951 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:43.462953 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:43.462970 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:43.463048 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:43.463322 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:43.464535 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:43.467210 (+ 2675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:43.468255 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:43.468318 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:43.468369 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:43.469254 (+ 885us) server_negotiation.cc:299] Negotiation successful
0811 05:12:43.469308 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":98,"threads_started":1}
I20260811 05:12:43.469588 30697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bab39b63a940426bb5d312f7ea9d8947. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:43.470122 30456 tablet_service.cc:1467] Processing CreateTablet for tablet bab39b63a940426bb5d312f7ea9d8947 (DEFAULT_TABLE table=test-table [id=8b231fd83dec47afa932df9b5b62b199]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:12:43.470456 30456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bab39b63a940426bb5d312f7ea9d8947. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:43.472147 31099 tablet_bootstrap.cc:492] T bab39b63a940426bb5d312f7ea9d8947 P 7aee251e414d44d98fad9f6bc76a7b9d: Bootstrap starting.
I20260811 05:12:43.472895 31099 tablet_bootstrap.cc:654] T bab39b63a940426bb5d312f7ea9d8947 P 7aee251e414d44d98fad9f6bc76a7b9d: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:43.473001 31100 tablet_bootstrap.cc:492] T bab39b63a940426bb5d312f7ea9d8947 P e9e36a61a9bc4e51ba87790b93fe9742: Bootstrap starting.
I20260811 05:12:43.473698 31100 tablet_bootstrap.cc:654] T bab39b63a940426bb5d312f7ea9d8947 P e9e36a61a9bc4e51ba87790b93fe9742: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:43.473798 31101 tablet_bootstrap.cc:492] T bab39b63a940426bb5d312f7ea9d8947 P 6a3f42398e9f43acaf35f9e4c953235a: Bootstrap starting.
I20260811 05:12:43.474048 31099 log.cc:824] T bab39b63a940426bb5d312f7ea9d8947 P 7aee251e414d44d98fad9f6bc76a7b9d: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:43.474264 31100 log.cc:824] T bab39b63a940426bb5d312f7ea9d8947 P e9e36a61a9bc4e51ba87790b93fe9742: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:43.474411 31101 tablet_bootstrap.cc:654] T bab39b63a940426bb5d312f7ea9d8947 P 6a3f42398e9f43acaf35f9e4c953235a: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:43.474916 31101 log.cc:824] T bab39b63a940426bb5d312f7ea9d8947 P 6a3f42398e9f43acaf35f9e4c953235a: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:43.475018 31099 tablet_bootstrap.cc:492] T bab39b63a940426bb5d312f7ea9d8947 P 7aee251e414d44d98fad9f6bc76a7b9d: No bootstrap required, opened a new log
I20260811 05:12:43.475101 31099 ts_tablet_manager.cc:1397] T bab39b63a940426bb5d312f7ea9d8947 P 7aee251e414d44d98fad9f6bc76a7b9d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 05:12:43.475344 31100 tablet_bootstrap.cc:492] T bab39b63a940426bb5d312f7ea9d8947 P e9e36a61a9bc4e51ba87790b93fe9742: No bootstrap required, opened a new log
I20260811 05:12:43.475479 31100 ts_tablet_manager.cc:1397] T bab39b63a940426bb5d312f7ea9d8947 P e9e36a61a9bc4e51ba87790b93fe9742: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 05:12:43.475747 31101 tablet_bootstrap.cc:492] T bab39b63a940426bb5d312f7ea9d8947 P 6a3f42398e9f43acaf35f9e4c953235a: No bootstrap required, opened a new log
I20260811 05:12:43.475819 31101 ts_tablet_manager.cc:1397] T bab39b63a940426bb5d312f7ea9d8947 P 6a3f42398e9f43acaf35f9e4c953235a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 05:12:43.477345 31101 raft_consensus.cc:348] T bab39b63a940426bb5d312f7ea9d8947 P 6a3f42398e9f43acaf35f9e4c953235a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aee251e414d44d98fad9f6bc76a7b9d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 46285 } } peers { permanent_uuid: "6a3f42398e9f43acaf35f9e4c953235a" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 45063 } } peers { permanent_uuid: "e9e36a61a9bc4e51ba87790b93fe9742" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34117 } }
I20260811 05:12:43.477506 31101 raft_consensus.cc:374] T bab39b63a940426bb5d312f7ea9d8947 P 6a3f42398e9f43acaf35f9e4c953235a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:43.477530 31101 raft_consensus.cc:729] T bab39b63a940426bb5d312f7ea9d8947 P 6a3f42398e9f43acaf35f9e4c953235a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a3f42398e9f43acaf35f9e4c953235a, State: Initialized, Role: FOLLOWER
I20260811 05:12:43.477272 31099 raft_consensus.cc:348] T bab39b63a940426bb5d312f7ea9d8947 P 7aee251e414d44d98fad9f6bc76a7b9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aee251e414d44d98fad9f6bc76a7b9d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 46285 } } peers { permanent_uuid: "6a3f42398e9f43acaf35f9e4c953235a" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 45063 } } peers { permanent_uuid: "e9e36a61a9bc4e51ba87790b93fe9742" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34117 } }
I20260811 05:12:43.477629 31099 raft_consensus.cc:374] T bab39b63a940426bb5d312f7ea9d8947 P 7aee251e414d44d98fad9f6bc76a7b9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:43.477643 31101 consensus_queue.cc:260] T bab39b63a940426bb5d312f7ea9d8947 P 6a3f42398e9f43acaf35f9e4c953235a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aee251e414d44d98fad9f6bc76a7b9d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 46285 } } peers { permanent_uuid: "6a3f42398e9f43acaf35f9e4c953235a" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 45063 } } peers { permanent_uuid: "e9e36a61a9bc4e51ba87790b93fe9742" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34117 } }
I20260811 05:12:43.477681 31099 raft_consensus.cc:729] T bab39b63a940426bb5d312f7ea9d8947 P 7aee251e414d44d98fad9f6bc76a7b9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aee251e414d44d98fad9f6bc76a7b9d, State: Initialized, Role: FOLLOWER
I20260811 05:12:43.477800 31099 consensus_queue.cc:260] T bab39b63a940426bb5d312f7ea9d8947 P 7aee251e414d44d98fad9f6bc76a7b9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aee251e414d44d98fad9f6bc76a7b9d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 46285 } } peers { permanent_uuid: "6a3f42398e9f43acaf35f9e4c953235a" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 45063 } } peers { permanent_uuid: "e9e36a61a9bc4e51ba87790b93fe9742" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34117 } }
I20260811 05:12:43.477944 30579 heartbeater.cc:503] Master 127.28.143.190:45963 was elected leader, sending a full tablet report...
I20260811 05:12:43.478116 31099 ts_tablet_manager.cc:1428] T bab39b63a940426bb5d312f7ea9d8947 P 7aee251e414d44d98fad9f6bc76a7b9d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 05:12:43.478129 31077 heartbeater.cc:503] Master 127.28.143.190:45963 was elected leader, sending a full tablet report...
I20260811 05:12:43.478230 31101 ts_tablet_manager.cc:1428] T bab39b63a940426bb5d312f7ea9d8947 P 6a3f42398e9f43acaf35f9e4c953235a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 05:12:43.478511 31100 raft_consensus.cc:348] T bab39b63a940426bb5d312f7ea9d8947 P e9e36a61a9bc4e51ba87790b93fe9742 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aee251e414d44d98fad9f6bc76a7b9d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 46285 } } peers { permanent_uuid: "6a3f42398e9f43acaf35f9e4c953235a" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 45063 } } peers { permanent_uuid: "e9e36a61a9bc4e51ba87790b93fe9742" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34117 } }
I20260811 05:12:43.478700 31100 raft_consensus.cc:374] T bab39b63a940426bb5d312f7ea9d8947 P e9e36a61a9bc4e51ba87790b93fe9742 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:43.478745 31100 raft_consensus.cc:729] T bab39b63a940426bb5d312f7ea9d8947 P e9e36a61a9bc4e51ba87790b93fe9742 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9e36a61a9bc4e51ba87790b93fe9742, State: Initialized, Role: FOLLOWER
I20260811 05:12:43.478861 31100 consensus_queue.cc:260] T bab39b63a940426bb5d312f7ea9d8947 P e9e36a61a9bc4e51ba87790b93fe9742 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aee251e414d44d98fad9f6bc76a7b9d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 46285 } } peers { permanent_uuid: "6a3f42398e9f43acaf35f9e4c953235a" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 45063 } } peers { permanent_uuid: "e9e36a61a9bc4e51ba87790b93fe9742" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34117 } }
I20260811 05:12:43.479194 31100 ts_tablet_manager.cc:1428] T bab39b63a940426bb5d312f7ea9d8947 P e9e36a61a9bc4e51ba87790b93fe9742: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260811 05:12:43.479365 30816 heartbeater.cc:503] Master 127.28.143.190:45963 was elected leader, sending a full tablet report...
W20260811 05:12:43.493949 30819 tablet.cc:2369] T bab39b63a940426bb5d312f7ea9d8947 P e9e36a61a9bc4e51ba87790b93fe9742: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 05:12:43.574041 30580 tablet.cc:2369] T bab39b63a940426bb5d312f7ea9d8947 P 6a3f42398e9f43acaf35f9e4c953235a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 05:12:43.680173 31078 tablet.cc:2369] T bab39b63a940426bb5d312f7ea9d8947 P 7aee251e414d44d98fad9f6bc76a7b9d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:12:43.690653 31105 raft_consensus.cc:482] T bab39b63a940426bb5d312f7ea9d8947 P 6a3f42398e9f43acaf35f9e4c953235a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:12:43.690797 31105 raft_consensus.cc:504] T bab39b63a940426bb5d312f7ea9d8947 P 6a3f42398e9f43acaf35f9e4c953235a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aee251e414d44d98fad9f6bc76a7b9d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 46285 } } peers { permanent_uuid: "6a3f42398e9f43acaf35f9e4c953235a" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 45063 } } peers { permanent_uuid: "e9e36a61a9bc4e51ba87790b93fe9742" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34117 } }
I20260811 05:12:43.691344 31105 leader_election.cc:288] T bab39b63a940426bb5d312f7ea9d8947 P 6a3f42398e9f43acaf35f9e4c953235a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aee251e414d44d98fad9f6bc76a7b9d (127.28.143.131:46285), e9e36a61a9bc4e51ba87790b93fe9742 (127.28.143.130:34117)
I20260811 05:12:43.694552 31094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:43.691641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:54115 (local address 127.28.143.131:46285)
0811 05:12:43.691689 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:43.691692 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:43.691896 (+ 204us) server_negotiation.cc:407] Connection header received
0811 05:12:43.692010 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:43.692011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:43.692020 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:43.692049 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:43.692307 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:43.692805 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:43.693835 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:43.694221 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:43.694290 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:43.694299 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:43.694432 (+ 133us) server_negotiation.cc:299] Negotiation successful
0811 05:12:43.694486 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 05:12:43.694552 31098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:43.691637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:37063 (local address 127.28.143.130:34117)
0811 05:12:43.691688 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:43.691690 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:43.691786 (+ 96us) server_negotiation.cc:407] Connection header received
0811 05:12:43.691842 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:43.691843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:43.691852 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:43.691967 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:43.692240 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:43.692796 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:43.693831 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:43.694204 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:43.694263 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:43.694273 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:43.694434 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 05:12:43.694475 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 05:12:43.694578 31109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:43.691543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:34117 (local address 127.28.143.129:37063)
0811 05:12:43.691709 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:43.691719 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:43.691784 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:43.691984 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:43.691985 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:43.691988 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:43.692073 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:43.692076 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:43.692873 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:43.692875 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:43.693735 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:43.693738 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:43.694283 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:43.694284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:43.694363 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:43.694367 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:12:43.694417 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 05:12:43.694462 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":100,"threads_started":1}
I20260811 05:12:43.694592 31114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:43.691544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.131:46285 (local address 127.28.143.129:54115)
0811 05:12:43.691804 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:43.691815 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:43.691927 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:43.692115 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:43.692116 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:43.692117 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:43.692208 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:43.692210 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:43.692893 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:43.692894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:43.693734 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:43.693736 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:43.694301 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:43.694302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:43.694362 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:43.694366 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:12:43.694414 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 05:12:43.694466 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"mutex_wait_us":1,"thread_start_us":97,"threads_started":1}
I20260811 05:12:43.695021 30980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bab39b63a940426bb5d312f7ea9d8947" candidate_uuid: "6a3f42398e9f43acaf35f9e4c953235a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aee251e414d44d98fad9f6bc76a7b9d" is_pre_election: true
I20260811 05:12:43.695217 30980 raft_consensus.cc:2454] T bab39b63a940426bb5d312f7ea9d8947 P 7aee251e414d44d98fad9f6bc76a7b9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a3f42398e9f43acaf35f9e4c953235a in term 0.
I20260811 05:12:43.695567 30379 leader_election.cc:302] T bab39b63a940426bb5d312f7ea9d8947 P 6a3f42398e9f43acaf35f9e4c953235a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a3f42398e9f43acaf35f9e4c953235a, 7aee251e414d44d98fad9f6bc76a7b9d; no voters:
I20260811 05:12:43.695744 31105 raft_consensus.cc:2781] T bab39b63a940426bb5d312f7ea9d8947 P 6a3f42398e9f43acaf35f9e4c953235a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:12:43.695817 31105 raft_consensus.cc:482] T bab39b63a940426bb5d312f7ea9d8947 P 6a3f42398e9f43acaf35f9e4c953235a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:12:43.695835 31105 raft_consensus.cc:3037] T bab39b63a940426bb5d312f7ea9d8947 P 6a3f42398e9f43acaf35f9e4c953235a [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:43.696591 31105 raft_consensus.cc:504] T bab39b63a940426bb5d312f7ea9d8947 P 6a3f42398e9f43acaf35f9e4c953235a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aee251e414d44d98fad9f6bc76a7b9d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 46285 } } peers { permanent_uuid: "6a3f42398e9f43acaf35f9e4c953235a" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 45063 } } peers { permanent_uuid: "e9e36a61a9bc4e51ba87790b93fe9742" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34117 } }
I20260811 05:12:43.696770 31105 leader_election.cc:288] T bab39b63a940426bb5d312f7ea9d8947 P 6a3f42398e9f43acaf35f9e4c953235a [CANDIDATE]: Term 1 election: Requested vote from peers 7aee251e414d44d98fad9f6bc76a7b9d (127.28.143.131:46285), e9e36a61a9bc4e51ba87790b93fe9742 (127.28.143.130:34117)
I20260811 05:12:43.696933 30980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bab39b63a940426bb5d312f7ea9d8947" candidate_uuid: "6a3f42398e9f43acaf35f9e4c953235a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aee251e414d44d98fad9f6bc76a7b9d"
I20260811 05:12:43.697038 30980 raft_consensus.cc:3037] T bab39b63a940426bb5d312f7ea9d8947 P 7aee251e414d44d98fad9f6bc76a7b9d [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:43.697005 30731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bab39b63a940426bb5d312f7ea9d8947" candidate_uuid: "6a3f42398e9f43acaf35f9e4c953235a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9e36a61a9bc4e51ba87790b93fe9742"
I20260811 05:12:43.697016 30733 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bab39b63a940426bb5d312f7ea9d8947" candidate_uuid: "6a3f42398e9f43acaf35f9e4c953235a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9e36a61a9bc4e51ba87790b93fe9742" is_pre_election: true
I20260811 05:12:43.697232 30731 raft_consensus.cc:3037] T bab39b63a940426bb5d312f7ea9d8947 P e9e36a61a9bc4e51ba87790b93fe9742 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:43.698004 30980 raft_consensus.cc:2454] T bab39b63a940426bb5d312f7ea9d8947 P 7aee251e414d44d98fad9f6bc76a7b9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a3f42398e9f43acaf35f9e4c953235a in term 1.
I20260811 05:12:43.698113 30731 raft_consensus.cc:2454] T bab39b63a940426bb5d312f7ea9d8947 P e9e36a61a9bc4e51ba87790b93fe9742 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a3f42398e9f43acaf35f9e4c953235a in term 1.
I20260811 05:12:43.698268 30379 leader_election.cc:302] T bab39b63a940426bb5d312f7ea9d8947 P 6a3f42398e9f43acaf35f9e4c953235a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a3f42398e9f43acaf35f9e4c953235a, 7aee251e414d44d98fad9f6bc76a7b9d; no voters:
I20260811 05:12:43.698407 31105 raft_consensus.cc:2781] T bab39b63a940426bb5d312f7ea9d8947 P 6a3f42398e9f43acaf35f9e4c953235a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:43.698570 31105 raft_consensus.cc:686] T bab39b63a940426bb5d312f7ea9d8947 P 6a3f42398e9f43acaf35f9e4c953235a [term 1 LEADER]: Becoming Leader. State: Replica: 6a3f42398e9f43acaf35f9e4c953235a, State: Running, Role: LEADER
I20260811 05:12:43.698652 31105 consensus_queue.cc:237] T bab39b63a940426bb5d312f7ea9d8947 P 6a3f42398e9f43acaf35f9e4c953235a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aee251e414d44d98fad9f6bc76a7b9d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 46285 } } peers { permanent_uuid: "6a3f42398e9f43acaf35f9e4c953235a" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 45063 } } peers { permanent_uuid: "e9e36a61a9bc4e51ba87790b93fe9742" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34117 } }
I20260811 05:12:43.699587 30219 catalog_manager.cc:5495] T bab39b63a940426bb5d312f7ea9d8947 P 6a3f42398e9f43acaf35f9e4c953235a reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a3f42398e9f43acaf35f9e4c953235a (127.28.143.129). New cstate: current_term: 1 leader_uuid: "6a3f42398e9f43acaf35f9e4c953235a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aee251e414d44d98fad9f6bc76a7b9d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 46285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a3f42398e9f43acaf35f9e4c953235a" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 45063 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9e36a61a9bc4e51ba87790b93fe9742" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34117 } health_report { overall_health: UNKNOWN } } }
Aug 11 05:12:43 dist-test-slave-f7th krb5kdc[30136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425162, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.143.129@KRBTEST.COM
I20260811 05:12:43.758570 31097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:43.750147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60146 (local address 127.28.143.129:45063)
0811 05:12:43.750208 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:43.750209 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:43.750301 (+ 92us) server_negotiation.cc:407] Connection header received
0811 05:12:43.750372 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:43.750374 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:43.750384 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:43.750435 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:43.751012 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:43.751556 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:43.752297 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:43.752535 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:43.752602 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:43.755239 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:43.755257 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:43.755257 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:43.755267 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:43.757142 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:43.757591 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:43.757595 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:43.757595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:43.757674 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:43.757963 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:43.757964 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:43.757964 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:43.758141 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:43.758342 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:43.758383 (+ 41us) server_negotiation.cc:299] Negotiation successful
0811 05:12:43.758486 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
Aug 11 05:12:43 dist-test-slave-f7th krb5kdc[30136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425163, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 05:12:43 dist-test-slave-f7th krb5kdc[30136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425163, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.28.143.129@KRBTEST.COM
I20260811 05:12:43.786550 31097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:43.779233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60152 (local address 127.28.143.129:45063)
0811 05:12:43.779291 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:43.779293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:43.779303 (+ 10us) server_negotiation.cc:407] Connection header received
0811 05:12:43.779419 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:43.779420 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:43.779432 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:43.779494 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:43.780053 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:43.780564 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:43.781348 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:43.781560 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:43.781621 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:43.783916 (+ 2295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:43.783917 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:43.783917 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:43.783925 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:43.785345 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:43.785750 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:43.785752 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:43.785752 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:43.785811 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:43.785983 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:43.785984 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:43.785984 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:43.786098 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:43.786193 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:43.786367 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 05:12:43.786455 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
Aug 11 05:12:43 dist-test-slave-f7th krb5kdc[30136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425163, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 05:12:43 dist-test-slave-f7th krb5kdc[30136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425163, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.143.129@KRBTEST.COM
I20260811 05:12:43.811640 31097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:43.803987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60168 (local address 127.28.143.129:45063)
0811 05:12:43.804079 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:43.804081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:43.804091 (+ 10us) server_negotiation.cc:407] Connection header received
0811 05:12:43.804117 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:43.804118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:43.804129 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:43.804186 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:43.804876 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:43.805356 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:43.806126 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:43.806337 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:43.806391 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:43.808934 (+ 2543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:43.808935 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:43.808935 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:43.808943 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:43.810359 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:43.810784 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:43.810785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:43.810786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:43.810848 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:43.811154 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:43.811156 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:43.811156 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:43.811268 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:43.811369 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:43.811485 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 05:12:43.811555 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260811 05:12:43.812569 30427 tablet_service.cc:2908] Rejecting scan request for tablet bab39b63a940426bb5d312f7ea9d8947: Uninitialized: clean time has not yet been initialized
I20260811 05:12:43.813789 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 30339
I20260811 05:12:43.835111 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 30589
I20260811 05:12:43.850168 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 30833
I20260811 05:12:43.870555 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 30166
2026-08-11T05:12:43Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2535 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:12:43 dist-test-slave-f7th krb5kdc[31141](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:12:43 dist-test-slave-f7th krb5kdc[31141](info): set up 2 sockets
Aug 11 05:12:43 dist-test-slave-f7th krb5kdc[31141](info): commencing operation
krb5kdc: starting...
W20260811 05:12:45.102360 29246 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Aug 11 05:12:45 dist-test-slave-f7th krb5kdc[31141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:12:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:12:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
I20260811 05:12:45.247757 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:34053
--webserver_interface=127.28.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:41649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:34053
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:45.336252 31167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:45.336457 31167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:45.336478 31167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:45.337663 31167 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:12:45.337714 31167 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 05:12:45.337728 31167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:45.337741 31167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:45.337757 31167 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:12:45.337774 31167 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:12:45.339303 31167 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:41649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:34053
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:34053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:12:45.339504 31167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:45.339680 31167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:45.344497 31176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:45.344501 31183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:45.344538 31179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:45.344846 31167 server_base.cc:1034] running on GCE node
I20260811 05:12:45.345139 31167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:45.345465 31167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:45.346647 31167 hybrid_clock.cc:648] HybridClock initialized: now 1786425165346629 us; error 31 us; skew 500 ppm
Aug 11 05:12:45 dist-test-slave-f7th krb5kdc[31141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425165, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:45.349344 31167 init.cc:377] Logged in from keytab as kudu/127.28.143.190@KRBTEST.COM (short username kudu)
I20260811 05:12:45.349830 31167 webserver.cc:460] Webserver started at http://127.28.143.190:44687/ using document root <none> and password file <none>
I20260811 05:12:45.350119 31167 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:45.350174 31167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:45.350308 31167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:45.351397 31167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "8798dcd480634809ac8bfa3f15e20049"
format_stamp: "Formatted at 2026-08-11 05:12:45 on dist-test-slave-f7th"
server_key: "5173b170675c884b768eed0eb0f058da"
server_key_iv: "3d884a76bc2426871a7b85df264f841e"
server_key_version: "encryptionkey@0"
I20260811 05:12:45.351728 31167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "8798dcd480634809ac8bfa3f15e20049"
format_stamp: "Formatted at 2026-08-11 05:12:45 on dist-test-slave-f7th"
server_key: "5173b170675c884b768eed0eb0f058da"
server_key_iv: "3d884a76bc2426871a7b85df264f841e"
server_key_version: "encryptionkey@0"
I20260811 05:12:45.353121 31167 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 05:12:45.353827 31198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:45.354045 31167 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:12:45.354148 31167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "8798dcd480634809ac8bfa3f15e20049"
format_stamp: "Formatted at 2026-08-11 05:12:45 on dist-test-slave-f7th"
server_key: "5173b170675c884b768eed0eb0f058da"
server_key_iv: "3d884a76bc2426871a7b85df264f841e"
server_key_version: "encryptionkey@0"
I20260811 05:12:45.354208 31167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:45.366786 31167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:45.369199 31167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:45.369352 31167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:45.374326 31167 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:34053
I20260811 05:12:45.374475 31311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:34053 every 8 connection(s)
I20260811 05:12:45.374796 31167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:12:45.375538 31313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:45.377925 31313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8798dcd480634809ac8bfa3f15e20049: Bootstrap starting.
I20260811 05:12:45.378512 31313 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8798dcd480634809ac8bfa3f15e20049: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:45.378741 31313 log.cc:824] T 00000000000000000000000000000000 P 8798dcd480634809ac8bfa3f15e20049: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:45.379410 31313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8798dcd480634809ac8bfa3f15e20049: No bootstrap required, opened a new log
I20260811 05:12:45.381791 31313 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8798dcd480634809ac8bfa3f15e20049 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8798dcd480634809ac8bfa3f15e20049" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 34053 } }
I20260811 05:12:45.381950 31313 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8798dcd480634809ac8bfa3f15e20049 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:45.381973 31313 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8798dcd480634809ac8bfa3f15e20049 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8798dcd480634809ac8bfa3f15e20049, State: Initialized, Role: FOLLOWER
I20260811 05:12:45.382112 31313 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8798dcd480634809ac8bfa3f15e20049 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8798dcd480634809ac8bfa3f15e20049" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 34053 } }
I20260811 05:12:45.382196 31313 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8798dcd480634809ac8bfa3f15e20049 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:12:45.382222 31313 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8798dcd480634809ac8bfa3f15e20049 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:12:45.382272 31313 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8798dcd480634809ac8bfa3f15e20049 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:45.382829 31313 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8798dcd480634809ac8bfa3f15e20049 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8798dcd480634809ac8bfa3f15e20049" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 34053 } }
I20260811 05:12:45.382925 31313 leader_election.cc:302] T 00000000000000000000000000000000 P 8798dcd480634809ac8bfa3f15e20049 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8798dcd480634809ac8bfa3f15e20049; no voters:
I20260811 05:12:45.383081 31313 leader_election.cc:288] T 00000000000000000000000000000000 P 8798dcd480634809ac8bfa3f15e20049 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:12:45.383141 31316 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8798dcd480634809ac8bfa3f15e20049 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:45.383311 31316 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8798dcd480634809ac8bfa3f15e20049 [term 1 LEADER]: Becoming Leader. State: Replica: 8798dcd480634809ac8bfa3f15e20049, State: Running, Role: LEADER
I20260811 05:12:45.383324 31313 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8798dcd480634809ac8bfa3f15e20049 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:12:45.383519 31316 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8798dcd480634809ac8bfa3f15e20049 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8798dcd480634809ac8bfa3f15e20049" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 34053 } }
I20260811 05:12:45.384068 31317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8798dcd480634809ac8bfa3f15e20049 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8798dcd480634809ac8bfa3f15e20049" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8798dcd480634809ac8bfa3f15e20049" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 34053 } } }
I20260811 05:12:45.384178 31317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8798dcd480634809ac8bfa3f15e20049 [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:45.384397 31333 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:12:45.384653 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 31167
I20260811 05:12:45.384758 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:12:45.384932 29246 external_mini_cluster.cc:1428] Setting key 7b599b5a4d76a2615ca4c7249ada72f0
I20260811 05:12:45.385190 31333 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:12:45.384424 31321 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8798dcd480634809ac8bfa3f15e20049 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8798dcd480634809ac8bfa3f15e20049. Latest consensus state: current_term: 1 leader_uuid: "8798dcd480634809ac8bfa3f15e20049" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8798dcd480634809ac8bfa3f15e20049" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 34053 } } }
I20260811 05:12:45.386255 31321 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8798dcd480634809ac8bfa3f15e20049 [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:45.387228 31333 catalog_manager.cc:1347] Generated new cluster ID: 727d47ab0ed24e1899abd99d5ad00711
I20260811 05:12:45.387287 31333 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 05:12:45 dist-test-slave-f7th krb5kdc[31141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:12:45.397262 31351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:45.385472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54824 (local address 127.28.143.190:34053)
0811 05:12:45.385672 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:45.385684 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:45.385718 (+ 34us) server_negotiation.cc:407] Connection header received
0811 05:12:45.385837 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:45.385843 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:45.385891 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:45.386020 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:45.386689 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:45.387800 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:45.389766 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:45.390270 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:45.390391 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:45.393104 (+ 2713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:45.393109 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:45.393110 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:45.393125 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:45.395152 (+ 2027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:45.395592 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:45.395596 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:45.395596 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:45.395744 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:45.395927 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:45.395928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:45.395929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:45.396271 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:45.396436 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:45.396523 (+ 87us) server_negotiation.cc:299] Negotiation successful
0811 05:12:45.396726 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":79,"threads_started":1}
I20260811 05:12:45.406116 31333 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:12:45.406679 31333 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:12:45.421595 31333 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8798dcd480634809ac8bfa3f15e20049: Generated new TSK 0
I20260811 05:12:45.421793 31333 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.143.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab.
Entry for principal kudu/127.28.143.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab.
I20260811 05:12:45.461395 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.129:0
--local_ip_for_outbound_sockets=127.28.143.129
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:34053
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:41649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:45.570779 31363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:45.571015 31363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:45.571043 31363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:45.572386 31363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:45.572450 31363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:45.572474 31363 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 05:12:45.572490 31363 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.129
I20260811 05:12:45.574347 31363 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:41649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.143.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.143.190:34053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.129
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:12:45.574644 31363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:45.574921 31363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:45.580389 31373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:45.580389 31371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:45.580407 31363 server_base.cc:1034] running on GCE node
W20260811 05:12:45.580387 31370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:45.580892 31363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:45.581187 31363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:45.582366 31363 hybrid_clock.cc:648] HybridClock initialized: now 1786425165582350 us; error 27 us; skew 500 ppm
Aug 11 05:12:45 dist-test-slave-f7th krb5kdc[31141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425165, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:45.589368 31363 init.cc:377] Logged in from keytab as kudu/127.28.143.129@KRBTEST.COM (short username kudu)
I20260811 05:12:45.589912 31363 webserver.cc:460] Webserver started at http://127.28.143.129:35331/ using document root <none> and password file <none>
I20260811 05:12:45.590163 31363 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:45.590217 31363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:45.590355 31363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:45.591595 31363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-0/data/instance:
uuid: "36cb15aca8b54ba9b7509260d9483543"
format_stamp: "Formatted at 2026-08-11 05:12:45 on dist-test-slave-f7th"
server_key: "e0844d387c3bd94e8c325c3637af9580"
server_key_iv: "990a8d2213022609e18098b39bc492d1"
server_key_version: "encryptionkey@0"
I20260811 05:12:45.592088 31363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance:
uuid: "36cb15aca8b54ba9b7509260d9483543"
format_stamp: "Formatted at 2026-08-11 05:12:45 on dist-test-slave-f7th"
server_key: "e0844d387c3bd94e8c325c3637af9580"
server_key_iv: "990a8d2213022609e18098b39bc492d1"
server_key_version: "encryptionkey@0"
I20260811 05:12:45.593851 31363 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260811 05:12:45.594831 31386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:45.595119 31363 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 05:12:45.595242 31363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-0/wal
uuid: "36cb15aca8b54ba9b7509260d9483543"
format_stamp: "Formatted at 2026-08-11 05:12:45 on dist-test-slave-f7th"
server_key: "e0844d387c3bd94e8c325c3637af9580"
server_key_iv: "990a8d2213022609e18098b39bc492d1"
server_key_version: "encryptionkey@0"
I20260811 05:12:45.595319 31363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:45.614758 31363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:45.617386 31363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:45.617568 31363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:45.618163 31363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:12:45.618209 31363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:45.618242 31363 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:12:45.618257 31363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 05:12:45 dist-test-slave-f7th krb5kdc[31141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425165, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:12:45.626358 31363 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.129:36521
I20260811 05:12:45.626413 31617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.129:36521 every 8 connection(s)
I20260811 05:12:45.626806 31363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
I20260811 05:12:45.629179 31351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:45.618843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:59225 (local address 127.28.143.190:34053)
0811 05:12:45.618921 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:45.618923 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:45.619170 (+ 247us) server_negotiation.cc:407] Connection header received
0811 05:12:45.619915 (+ 745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:45.619917 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:45.619938 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:45.620013 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:45.621125 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:45.621699 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:45.623061 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:45.623277 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:45.623351 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:45.626299 (+ 2948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:45.626302 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:45.626302 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:45.626313 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:45.627891 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:45.628326 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:45.628326 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:45.628327 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:45.628378 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:45.628567 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:45.628568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:45.628568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:45.628725 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:45.628782 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:45.629032 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 05:12:45.629109 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 05:12:45.629362 31409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:45.618794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:34053 (local address 127.28.143.129:59225)
0811 05:12:45.619001 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:45.619062 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:45.619767 (+ 705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:45.620094 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:45.620098 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:45.620633 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:45.620958 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:45.620967 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:45.621980 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:45.621981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:45.622954 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:45.622957 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:45.623383 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:45.623385 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:45.623546 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:45.624119 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:12:45.624128 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:45.626141 (+ 2013us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:45.628007 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:45.628015 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:45.628017 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:45.628254 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:45.628423 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:45.628424 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:45.628424 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:45.628522 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:45.628779 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:45.628784 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:45.628916 (+ 132us) client_negotiation.cc:769] Sending connection context
0811 05:12:45.629005 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 05:12:45.629144 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":93,"threads_started":1}
I20260811 05:12:45.629580 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 31363
I20260811 05:12:45.629673 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance
I20260811 05:12:45.629854 29246 external_mini_cluster.cc:1428] Setting key caae67125611f364a618761c1d85bfaa
I20260811 05:12:45.629985 31620 heartbeater.cc:347] Connected to a master server at 127.28.143.190:34053
I20260811 05:12:45.630102 31620 heartbeater.cc:464] Registering TS with master...
I20260811 05:12:45.630328 31620 heartbeater.cc:511] Master 127.28.143.190:34053 requested a full tablet report, sending...
I20260811 05:12:45.631033 31226 ts_manager.cc:194] Registered new tserver with Master: 36cb15aca8b54ba9b7509260d9483543 (127.28.143.129:36521)
I20260811 05:12:45.631850 31226 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.129@KRBTEST.COM'} at 127.28.143.129:59225
I20260811 05:12:45.638142 31627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:45.631068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:34053 (local address 127.28.143.129:41611)
0811 05:12:45.631223 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:45.631232 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:45.631276 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:45.632234 (+ 958us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:45.632235 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:45.632457 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:45.632532 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:45.632535 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:45.633323 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:45.633324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:45.633814 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:45.633816 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:45.634242 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:45.634243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:45.634312 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:45.634792 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:12:45.634796 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:45.635111 (+ 315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:45.637072 (+ 1961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:45.637073 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:45.637073 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:45.637249 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:45.637490 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:45.637491 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:45.637491 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:45.637558 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:45.637823 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:45.637824 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:45.637876 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 05:12:45.637967 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 05:12:45.638061 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":73,"threads_started":1}
I20260811 05:12:45.638185 31351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:45.632045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:41611 (local address 127.28.143.190:34053)
0811 05:12:45.632115 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:45.632116 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:45.632125 (+ 9us) server_negotiation.cc:407] Connection header received
0811 05:12:45.632144 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:45.632144 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:45.632153 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:45.632206 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:45.632661 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:45.633227 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:45.633891 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:45.634152 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:45.634217 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:45.635202 (+ 985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:45.635204 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:45.635204 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:45.635213 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:45.636953 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:45.637323 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:45.637324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:45.637324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:45.637400 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:45.637625 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:45.637625 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:45.637626 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:45.637765 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:45.637835 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:45.637997 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 05:12:45.638090 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
WARNING: no policy specified for kudu/127.28.143.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab.
Entry for principal kudu/127.28.143.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab.
I20260811 05:12:45.660769 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.130:0
--local_ip_for_outbound_sockets=127.28.143.130
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:34053
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:41649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:45.745488 31633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:45.745707 31633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:45.745743 31633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:45.746956 31633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:45.747007 31633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:45.747042 31633 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 05:12:45.747068 31633 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.130
I20260811 05:12:45.748571 31633 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:41649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.143.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.143.190:34053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.130
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:12:45.748862 31633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:45.749094 31633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:45.754305 31649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:45.754312 31643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:45.754307 31642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:45.754575 31633 server_base.cc:1034] running on GCE node
I20260811 05:12:45.754787 31633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:45.755035 31633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:45.756258 31633 hybrid_clock.cc:648] HybridClock initialized: now 1786425165756235 us; error 40 us; skew 500 ppm
Aug 11 05:12:45 dist-test-slave-f7th krb5kdc[31141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425165, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:45.759336 31633 init.cc:377] Logged in from keytab as kudu/127.28.143.130@KRBTEST.COM (short username kudu)
I20260811 05:12:45.759815 31633 webserver.cc:460] Webserver started at http://127.28.143.130:37965/ using document root <none> and password file <none>
I20260811 05:12:45.760011 31633 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:45.760056 31633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:45.760147 31633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:45.761268 31633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-1/data/instance:
uuid: "59c38762b52b47b2a417ceb0cee2f0b1"
format_stamp: "Formatted at 2026-08-11 05:12:45 on dist-test-slave-f7th"
server_key: "07297c639c4ded50b86ebb0de3b99d9c"
server_key_iv: "04dfddd6e3e755cd59650c5f6c3d0040"
server_key_version: "encryptionkey@0"
I20260811 05:12:45.761616 31633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance:
uuid: "59c38762b52b47b2a417ceb0cee2f0b1"
format_stamp: "Formatted at 2026-08-11 05:12:45 on dist-test-slave-f7th"
server_key: "07297c639c4ded50b86ebb0de3b99d9c"
server_key_iv: "04dfddd6e3e755cd59650c5f6c3d0040"
server_key_version: "encryptionkey@0"
I20260811 05:12:45.763136 31633 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 05:12:45.764135 31666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:45.764328 31633 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:12:45.764425 31633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-1/wal
uuid: "59c38762b52b47b2a417ceb0cee2f0b1"
format_stamp: "Formatted at 2026-08-11 05:12:45 on dist-test-slave-f7th"
server_key: "07297c639c4ded50b86ebb0de3b99d9c"
server_key_iv: "04dfddd6e3e755cd59650c5f6c3d0040"
server_key_version: "encryptionkey@0"
I20260811 05:12:45.764524 31633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:45.774477 31633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:45.777259 31633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:45.777400 31633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:45.778064 31633 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:12:45.778112 31633 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:45.778136 31633 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:12:45.778147 31633 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:45.785315 31633 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.130:41625
I20260811 05:12:45.785429 31864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.130:41625 every 8 connection(s)
Aug 11 05:12:45 dist-test-slave-f7th krb5kdc[31141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425165, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:12:45.785729 31633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
I20260811 05:12:45.787072 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 31633
I20260811 05:12:45.787163 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance
I20260811 05:12:45.787546 29246 external_mini_cluster.cc:1428] Setting key 2d035649b667c77a92449127c993b7b6
I20260811 05:12:45.789496 31351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:45.778564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:59953 (local address 127.28.143.190:34053)
0811 05:12:45.778631 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:45.778634 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:45.778802 (+ 168us) server_negotiation.cc:407] Connection header received
0811 05:12:45.779292 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:45.779294 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:45.779307 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:45.779387 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:45.780218 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:45.781072 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:45.782668 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:45.783071 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:45.783147 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:45.786266 (+ 3119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:45.786268 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:45.786269 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:45.786279 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:45.787921 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:45.788383 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:45.788384 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:45.788385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:45.788467 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:45.788787 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:45.788788 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:45.788789 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:45.788932 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:45.789015 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:45.789320 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 05:12:45.789407 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 05:12:45.789681 31684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:45.778493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:34053 (local address 127.28.143.130:59953)
0811 05:12:45.778674 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:45.778717 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:45.779201 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:45.779466 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:45.779470 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:45.779816 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:45.780105 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:45.780111 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:45.781194 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:45.781196 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:45.782556 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:45.782560 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:45.783196 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:45.783198 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:45.783416 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:45.784120 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:12:45.784132 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:45.786124 (+ 1992us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:45.788001 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:45.788012 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:45.788015 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:45.788280 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:45.788555 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:45.788556 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:45.788556 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:45.788724 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:45.789036 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:45.789043 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:45.789183 (+ 140us) client_negotiation.cc:769] Sending connection context
0811 05:12:45.789286 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 05:12:45.789473 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":112,"threads_started":1}
I20260811 05:12:45.791229 31865 heartbeater.cc:347] Connected to a master server at 127.28.143.190:34053
I20260811 05:12:45.791328 31865 heartbeater.cc:464] Registering TS with master...
I20260811 05:12:45.791523 31865 heartbeater.cc:511] Master 127.28.143.190:34053 requested a full tablet report, sending...
I20260811 05:12:45.791942 31228 ts_manager.cc:194] Registered new tserver with Master: 59c38762b52b47b2a417ceb0cee2f0b1 (127.28.143.130:41625)
I20260811 05:12:45.792433 31228 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.130@KRBTEST.COM'} at 127.28.143.130:59953
WARNING: no policy specified for kudu/127.28.143.131@KRBTEST.COM; defaulting to no policy
I20260811 05:12:45.797549 31351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:45.791570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:35807 (local address 127.28.143.190:34053)
0811 05:12:45.791621 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:45.791622 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:45.791654 (+ 32us) server_negotiation.cc:407] Connection header received
0811 05:12:45.791751 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:45.791752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:45.791763 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:45.791825 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:45.792166 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:45.792738 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:45.793628 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:45.793830 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:45.793884 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:45.794832 (+ 948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:45.794833 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:45.794834 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:45.794841 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:45.796319 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:45.796716 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:45.796717 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:45.796718 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:45.796781 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:45.796951 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:45.796951 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:45.796952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:45.797115 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:45.797176 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:45.797373 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 05:12:45.797452 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 05:12:45.797562 31875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:45.791365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:34053 (local address 127.28.143.130:35807)
0811 05:12:45.791589 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:45.791600 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:45.791680 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:45.791834 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:45.791834 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:45.792027 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:45.792088 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:45.792090 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:45.792825 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:45.792826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:45.793541 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:45.793544 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:45.793892 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:45.793893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:45.793965 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:45.794411 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:12:45.794416 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:45.794743 (+ 327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:45.796425 (+ 1682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:45.796426 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:45.796427 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:45.796648 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:45.796836 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:45.796837 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:45.796837 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:45.796898 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:45.797178 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:45.797179 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:45.797228 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 05:12:45.797307 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 05:12:45.797423 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":77,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab.
Entry for principal kudu/127.28.143.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab.
I20260811 05:12:45.820318 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.131:0
--local_ip_for_outbound_sockets=127.28.143.131
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:34053
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:41649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:45.908268 31880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:45.908491 31880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:45.908516 31880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:45.909742 31880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:45.909789 31880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:45.909809 31880 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 05:12:45.909827 31880 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.131
I20260811 05:12:45.911347 31880 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:41649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.143.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.143.190:34053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31880
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.131
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:12:45.911585 31880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:45.911811 31880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:45.917026 31890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:45.917088 31889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:45.917091 31894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:45.917308 31880 server_base.cc:1034] running on GCE node
I20260811 05:12:45.917522 31880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:45.917776 31880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:45.918979 31880 hybrid_clock.cc:648] HybridClock initialized: now 1786425165918966 us; error 28 us; skew 500 ppm
Aug 11 05:12:45 dist-test-slave-f7th krb5kdc[31141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425165, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:45.921695 31880 init.cc:377] Logged in from keytab as kudu/127.28.143.131@KRBTEST.COM (short username kudu)
I20260811 05:12:45.922114 31880 webserver.cc:460] Webserver started at http://127.28.143.131:38703/ using document root <none> and password file <none>
I20260811 05:12:45.922328 31880 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:45.922370 31880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:45.922465 31880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:45.923522 31880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-2/data/instance:
uuid: "412b9bad9194413faeeda0d15b3cdb2d"
format_stamp: "Formatted at 2026-08-11 05:12:45 on dist-test-slave-f7th"
server_key: "0696be5f757cbe1c866b3b66170ad1a4"
server_key_iv: "1c4e5bcdca1a426ae47ee2c95b22cc6a"
server_key_version: "encryptionkey@0"
I20260811 05:12:45.923830 31880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance:
uuid: "412b9bad9194413faeeda0d15b3cdb2d"
format_stamp: "Formatted at 2026-08-11 05:12:45 on dist-test-slave-f7th"
server_key: "0696be5f757cbe1c866b3b66170ad1a4"
server_key_iv: "1c4e5bcdca1a426ae47ee2c95b22cc6a"
server_key_version: "encryptionkey@0"
I20260811 05:12:45.925166 31880 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 05:12:45.926148 31913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:45.926335 31880 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:12:45.926420 31880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-2/wal
uuid: "412b9bad9194413faeeda0d15b3cdb2d"
format_stamp: "Formatted at 2026-08-11 05:12:45 on dist-test-slave-f7th"
server_key: "0696be5f757cbe1c866b3b66170ad1a4"
server_key_iv: "1c4e5bcdca1a426ae47ee2c95b22cc6a"
server_key_version: "encryptionkey@0"
I20260811 05:12:45.926504 31880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:45.936009 31880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:45.938328 31880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:45.938458 31880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:45.938992 31880 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:12:45.939031 31880 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:45.939055 31880 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:12:45.939073 31880 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:45.945887 31880 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.131:44771
I20260811 05:12:45.945966 32119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.131:44771 every 8 connection(s)
I20260811 05:12:45.946277 31880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
Aug 11 05:12:45 dist-test-slave-f7th krb5kdc[31141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425165, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:12:45.947626 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 31880
I20260811 05:12:45.947716 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance
I20260811 05:12:45.947882 29246 external_mini_cluster.cc:1428] Setting key 2cbc94755f569436ac41114c3d20fb8e
I20260811 05:12:45.949622 31351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:45.939616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:37659 (local address 127.28.143.190:34053)
0811 05:12:45.939670 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:45.939672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:45.939915 (+ 243us) server_negotiation.cc:407] Connection header received
0811 05:12:45.940445 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:45.940446 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:45.940460 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:45.940521 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:45.941457 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:45.942206 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:45.943763 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:45.944044 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:45.944104 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:45.946875 (+ 2771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:45.946877 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:45.946877 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:45.946886 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:45.948238 (+ 1352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:45.948691 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:45.948692 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:45.948693 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:45.948757 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:45.948995 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:45.948996 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:45.948997 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:45.949115 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:45.949170 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:45.949448 (+ 278us) server_negotiation.cc:299] Negotiation successful
0811 05:12:45.949533 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 05:12:45.949790 31925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:45.939583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:34053 (local address 127.28.143.131:37659)
0811 05:12:45.939765 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:45.939804 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:45.940342 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:45.940669 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:45.940675 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:45.941029 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:45.941334 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:45.941339 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:45.942313 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:45.942314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:45.943661 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:45.943665 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:45.944134 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:45.944135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:45.944290 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:45.945001 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:12:45.945014 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:45.946774 (+ 1760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:45.948319 (+ 1545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:45.948327 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:45.948330 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:45.948585 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:45.948809 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:45.948809 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:45.948810 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:45.948923 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:45.949172 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:45.949179 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:45.949316 (+ 137us) client_negotiation.cc:769] Sending connection context
0811 05:12:45.949401 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 05:12:45.949571 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":91,"threads_started":1}
I20260811 05:12:45.950232 32120 heartbeater.cc:347] Connected to a master server at 127.28.143.190:34053
I20260811 05:12:45.950326 32120 heartbeater.cc:464] Registering TS with master...
I20260811 05:12:45.950537 32120 heartbeater.cc:511] Master 127.28.143.190:34053 requested a full tablet report, sending...
I20260811 05:12:45.951120 31226 ts_manager.cc:194] Registered new tserver with Master: 412b9bad9194413faeeda0d15b3cdb2d (127.28.143.131:44771)
I20260811 05:12:45.951563 31226 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.131@KRBTEST.COM'} at 127.28.143.131:37659
I20260811 05:12:45.957468 32127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:45.951258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:34053 (local address 127.28.143.131:46335)
0811 05:12:45.951719 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:45.951731 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:45.951796 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:45.952091 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:45.952092 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:45.952264 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:45.952324 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:45.952325 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:45.953041 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:45.953041 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:45.953581 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:45.953584 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:45.953979 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:45.953980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:45.954047 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:45.954506 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:12:45.954510 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:45.954843 (+ 333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:45.956449 (+ 1606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:45.956451 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:45.956451 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:45.956693 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:45.956896 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:45.956897 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:45.956897 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:45.956967 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:45.957198 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:45.957199 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:45.957253 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 05:12:45.957300 (+ 47us) client_negotiation.cc:240] Negotiation successful
0811 05:12:45.957388 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":72,"threads_started":1}
I20260811 05:12:45.957494 31351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:45.951291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:46335 (local address 127.28.143.190:34053)
0811 05:12:45.951326 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:45.951328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:45.951806 (+ 478us) server_negotiation.cc:407] Connection header received
0811 05:12:45.951970 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:45.951971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:45.951983 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:45.952052 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:45.952404 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:45.952936 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:45.953663 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:45.953846 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:45.953942 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:45.954959 (+ 1017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:45.954961 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:45.954961 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:45.954968 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:45.956333 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:45.956757 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:45.956758 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:45.956759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:45.956827 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:45.957018 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:45.957018 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:45.957019 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:45.957149 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:45.957202 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:45.957317 (+ 115us) server_negotiation.cc:299] Negotiation successful
0811 05:12:45.957383 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260811 05:12:45.959442 29246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 05:12:45 dist-test-slave-f7th krb5kdc[31141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425165, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 05:12:45 dist-test-slave-f7th krb5kdc[31141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425165, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:12:45.985198 31351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:45.976976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54838 (local address 127.28.143.190:34053)
0811 05:12:45.977063 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:45.977064 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:45.977076 (+ 12us) server_negotiation.cc:407] Connection header received
0811 05:12:45.977146 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:45.977147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:45.977156 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:45.977205 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:45.977811 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:45.978359 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:45.979142 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:45.979342 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:45.979406 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:45.982059 (+ 2653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:45.982061 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:45.982061 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:45.982070 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:45.983835 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:45.984308 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:45.984309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:45.984309 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:45.984376 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:45.984662 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:45.984664 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:45.984664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:45.984813 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:45.984900 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:45.985050 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 05:12:45.985125 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 05:12:45.987459 31226 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54838:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:12:45.989346 31226 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:12:45.997611 32153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:45.992843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56450 (local address 127.28.143.129:36521)
0811 05:12:45.993207 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:45.993213 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:45.993223 (+ 10us) server_negotiation.cc:407] Connection header received
0811 05:12:45.993275 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:45.993277 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:45.993290 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:45.993380 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:45.993946 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:45.994858 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:45.996491 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:45.997256 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:45.997321 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:45.997371 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:45.997471 (+ 100us) server_negotiation.cc:299] Negotiation successful
0811 05:12:45.997510 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":90,"threads_started":1}
I20260811 05:12:45.997588 32152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:45.992702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:41625 (local address 127.0.0.1:41018)
0811 05:12:45.992991 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:45.993041 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:45.993114 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:45.993587 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:45.993591 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:45.993594 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:45.993693 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:45.993697 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:45.995111 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:45.995112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:45.996524 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:45.996526 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:45.997225 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:45.997226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:45.997304 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:45.997309 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 05:12:45.997365 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 05:12:45.997450 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":242,"thread_start_us":70,"threads_started":1}
I20260811 05:12:45.997704 32157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:45.993197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49588 (local address 127.28.143.131:44771)
0811 05:12:45.993359 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:45.993363 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:45.993758 (+ 395us) server_negotiation.cc:407] Connection header received
0811 05:12:45.993852 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:45.993855 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:45.993871 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:45.993964 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:45.994068 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:45.995095 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:45.996270 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:45.997021 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:45.997096 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:45.997155 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:45.997551 (+ 396us) server_negotiation.cc:299] Negotiation successful
0811 05:12:45.997596 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":82,"threads_started":1}
I20260811 05:12:45.997699 32156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:45.993038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.131:44771 (local address 127.0.0.1:49588)
0811 05:12:45.993590 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:45.993626 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:45.993768 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:45.993963 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:45.993964 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:45.993966 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:45.994003 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:45.994004 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:45.995161 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:45.995163 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:45.996189 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:45.996192 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:45.997194 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:45.997195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:45.997278 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:45.997296 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 05:12:45.997363 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 05:12:45.997449 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":69,"threads_started":1}
I20260811 05:12:45.998296 32155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:45.992963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41018 (local address 127.28.143.130:41625)
0811 05:12:45.993403 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:45.993406 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:45.993415 (+ 9us) server_negotiation.cc:407] Connection header received
0811 05:12:45.993451 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:45.993453 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:45.993467 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:45.993779 (+ 312us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:45.993843 (+ 64us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:45.995005 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:45.996603 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:45.997139 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:45.997211 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:45.997265 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:45.998142 (+ 877us) server_negotiation.cc:299] Negotiation successful
0811 05:12:45.998191 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":123,"threads_started":1}
I20260811 05:12:45.997588 32150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:45.992823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:36521 (local address 127.0.0.1:56450)
0811 05:12:45.993122 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:45.993133 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:45.993187 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:45.993603 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:45.993607 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:45.993613 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:45.993847 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:45.993850 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:45.994991 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:45.994994 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:45.996377 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:45.996380 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:45.997344 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:45.997345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:45.997401 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:45.997405 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:12:45.997457 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 05:12:45.997489 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":73,"thread_start_us":104,"threads_started":1}
I20260811 05:12:45.998917 31761 tablet_service.cc:1467] Processing CreateTablet for tablet 4b48a97f6c6d422dab1a9ee2545bbbd5 (DEFAULT_TABLE table=test-table [id=48d6d2de2b3b49308b5d87f03ed9e91e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:12:45.998879 32002 tablet_service.cc:1467] Processing CreateTablet for tablet 4b48a97f6c6d422dab1a9ee2545bbbd5 (DEFAULT_TABLE table=test-table [id=48d6d2de2b3b49308b5d87f03ed9e91e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:12:45.999341 31761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b48a97f6c6d422dab1a9ee2545bbbd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:45.999341 32002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b48a97f6c6d422dab1a9ee2545bbbd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:46.001466 31501 tablet_service.cc:1467] Processing CreateTablet for tablet 4b48a97f6c6d422dab1a9ee2545bbbd5 (DEFAULT_TABLE table=test-table [id=48d6d2de2b3b49308b5d87f03ed9e91e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:12:46.001824 31501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b48a97f6c6d422dab1a9ee2545bbbd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:46.003001 32158 tablet_bootstrap.cc:492] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 412b9bad9194413faeeda0d15b3cdb2d: Bootstrap starting.
I20260811 05:12:46.003347 32160 tablet_bootstrap.cc:492] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 59c38762b52b47b2a417ceb0cee2f0b1: Bootstrap starting.
I20260811 05:12:46.004133 32160 tablet_bootstrap.cc:654] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 59c38762b52b47b2a417ceb0cee2f0b1: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:46.004499 32160 log.cc:824] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 59c38762b52b47b2a417ceb0cee2f0b1: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:46.004812 32158 tablet_bootstrap.cc:654] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 412b9bad9194413faeeda0d15b3cdb2d: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:46.005174 32158 log.cc:824] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 412b9bad9194413faeeda0d15b3cdb2d: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:46.005342 32160 tablet_bootstrap.cc:492] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 59c38762b52b47b2a417ceb0cee2f0b1: No bootstrap required, opened a new log
I20260811 05:12:46.005474 32160 ts_tablet_manager.cc:1397] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 59c38762b52b47b2a417ceb0cee2f0b1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 05:12:46.005888 32169 tablet_bootstrap.cc:492] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 36cb15aca8b54ba9b7509260d9483543: Bootstrap starting.
I20260811 05:12:46.006029 32158 tablet_bootstrap.cc:492] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 412b9bad9194413faeeda0d15b3cdb2d: No bootstrap required, opened a new log
I20260811 05:12:46.006109 32158 ts_tablet_manager.cc:1397] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 412b9bad9194413faeeda0d15b3cdb2d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 05:12:46.006489 32169 tablet_bootstrap.cc:654] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 36cb15aca8b54ba9b7509260d9483543: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:46.006803 32169 log.cc:824] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 36cb15aca8b54ba9b7509260d9483543: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:46.007467 32169 tablet_bootstrap.cc:492] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 36cb15aca8b54ba9b7509260d9483543: No bootstrap required, opened a new log
I20260811 05:12:46.007536 32169 ts_tablet_manager.cc:1397] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 36cb15aca8b54ba9b7509260d9483543: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 05:12:46.007604 32160 raft_consensus.cc:348] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 59c38762b52b47b2a417ceb0cee2f0b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36cb15aca8b54ba9b7509260d9483543" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36521 } } peers { permanent_uuid: "412b9bad9194413faeeda0d15b3cdb2d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44771 } } peers { permanent_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41625 } }
I20260811 05:12:46.007795 32160 raft_consensus.cc:374] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 59c38762b52b47b2a417ceb0cee2f0b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:46.007844 32160 raft_consensus.cc:729] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 59c38762b52b47b2a417ceb0cee2f0b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59c38762b52b47b2a417ceb0cee2f0b1, State: Initialized, Role: FOLLOWER
I20260811 05:12:46.007974 32160 consensus_queue.cc:260] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 59c38762b52b47b2a417ceb0cee2f0b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36cb15aca8b54ba9b7509260d9483543" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36521 } } peers { permanent_uuid: "412b9bad9194413faeeda0d15b3cdb2d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44771 } } peers { permanent_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41625 } }
I20260811 05:12:46.008265 32160 ts_tablet_manager.cc:1428] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 59c38762b52b47b2a417ceb0cee2f0b1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 05:12:46.008196 32158 raft_consensus.cc:348] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 412b9bad9194413faeeda0d15b3cdb2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36cb15aca8b54ba9b7509260d9483543" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36521 } } peers { permanent_uuid: "412b9bad9194413faeeda0d15b3cdb2d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44771 } } peers { permanent_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41625 } }
I20260811 05:12:46.008297 31865 heartbeater.cc:503] Master 127.28.143.190:34053 was elected leader, sending a full tablet report...
I20260811 05:12:46.008358 32158 raft_consensus.cc:374] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 412b9bad9194413faeeda0d15b3cdb2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:46.008401 32158 raft_consensus.cc:729] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 412b9bad9194413faeeda0d15b3cdb2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 412b9bad9194413faeeda0d15b3cdb2d, State: Initialized, Role: FOLLOWER
I20260811 05:12:46.008512 32158 consensus_queue.cc:260] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 412b9bad9194413faeeda0d15b3cdb2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36cb15aca8b54ba9b7509260d9483543" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36521 } } peers { permanent_uuid: "412b9bad9194413faeeda0d15b3cdb2d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44771 } } peers { permanent_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41625 } }
I20260811 05:12:46.008805 32158 ts_tablet_manager.cc:1428] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 412b9bad9194413faeeda0d15b3cdb2d: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 05:12:46.008940 32120 heartbeater.cc:503] Master 127.28.143.190:34053 was elected leader, sending a full tablet report...
I20260811 05:12:46.010362 32169 raft_consensus.cc:348] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 36cb15aca8b54ba9b7509260d9483543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36cb15aca8b54ba9b7509260d9483543" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36521 } } peers { permanent_uuid: "412b9bad9194413faeeda0d15b3cdb2d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44771 } } peers { permanent_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41625 } }
I20260811 05:12:46.010499 32169 raft_consensus.cc:374] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 36cb15aca8b54ba9b7509260d9483543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:46.010543 32169 raft_consensus.cc:729] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 36cb15aca8b54ba9b7509260d9483543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36cb15aca8b54ba9b7509260d9483543, State: Initialized, Role: FOLLOWER
I20260811 05:12:46.010660 32169 consensus_queue.cc:260] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 36cb15aca8b54ba9b7509260d9483543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36cb15aca8b54ba9b7509260d9483543" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36521 } } peers { permanent_uuid: "412b9bad9194413faeeda0d15b3cdb2d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44771 } } peers { permanent_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41625 } }
I20260811 05:12:46.010896 32169 ts_tablet_manager.cc:1428] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 36cb15aca8b54ba9b7509260d9483543: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 05:12:46.010968 31620 heartbeater.cc:503] Master 127.28.143.190:34053 was elected leader, sending a full tablet report...
W20260811 05:12:46.036418 31866 tablet.cc:2369] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 59c38762b52b47b2a417ceb0cee2f0b1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 05:12:46.127758 31621 tablet.cc:2369] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 36cb15aca8b54ba9b7509260d9483543: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 05:12:46.196954 32121 tablet.cc:2369] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 412b9bad9194413faeeda0d15b3cdb2d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:12:46.246886 32176 raft_consensus.cc:482] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 59c38762b52b47b2a417ceb0cee2f0b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:12:46.247042 32176 raft_consensus.cc:504] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 59c38762b52b47b2a417ceb0cee2f0b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36cb15aca8b54ba9b7509260d9483543" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36521 } } peers { permanent_uuid: "412b9bad9194413faeeda0d15b3cdb2d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44771 } } peers { permanent_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41625 } }
I20260811 05:12:46.247355 32176 leader_election.cc:288] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 59c38762b52b47b2a417ceb0cee2f0b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 36cb15aca8b54ba9b7509260d9483543 (127.28.143.129:36521), 412b9bad9194413faeeda0d15b3cdb2d (127.28.143.131:44771)
I20260811 05:12:46.250566 32153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:46.247674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:39367 (local address 127.28.143.129:36521)
0811 05:12:46.247731 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:46.247734 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:46.247744 (+ 10us) server_negotiation.cc:407] Connection header received
0811 05:12:46.247827 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:46.247828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:46.247838 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:46.247948 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:46.248156 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:46.248753 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:46.249861 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:46.250225 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:46.250282 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:46.250291 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:46.250452 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 05:12:46.250496 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 05:12:46.250551 31684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:46.247602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:36521 (local address 127.28.143.130:39367)
0811 05:12:46.247669 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:46.247681 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:46.247751 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:46.247963 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:46.247964 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:46.247966 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:46.248045 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:46.248047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:46.248851 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:46.248853 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:46.249773 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:46.249776 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:46.250307 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:46.250308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:46.250386 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:46.250391 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 05:12:46.250433 (+ 42us) client_negotiation.cc:240] Negotiation successful
0811 05:12:46.250470 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 05:12:46.250593 32157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:46.247967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:33873 (local address 127.28.143.131:44771)
0811 05:12:46.248029 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:46.248031 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:46.248044 (+ 13us) server_negotiation.cc:407] Connection header received
0811 05:12:46.248069 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:46.248070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:46.248078 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:46.248132 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:46.248376 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:46.248933 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:46.249902 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:46.250263 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:46.250334 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:46.250343 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:46.250495 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 05:12:46.250537 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 05:12:46.250677 32179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:46.247721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.131:44771 (local address 127.28.143.130:33873)
0811 05:12:46.247879 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:46.247890 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:46.247949 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:46.248167 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:46.248168 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:46.248171 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:46.248246 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:46.248248 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:46.249010 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:46.249011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:46.249786 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:46.249788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:46.250342 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:46.250343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:46.250403 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:46.250406 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 05:12:46.250464 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 05:12:46.250539 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":2,"thread_start_us":94,"threads_started":1}
I20260811 05:12:46.250969 31535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b48a97f6c6d422dab1a9ee2545bbbd5" candidate_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36cb15aca8b54ba9b7509260d9483543" is_pre_election: true
I20260811 05:12:46.250982 32034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b48a97f6c6d422dab1a9ee2545bbbd5" candidate_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "412b9bad9194413faeeda0d15b3cdb2d" is_pre_election: true
I20260811 05:12:46.251139 32034 raft_consensus.cc:2454] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 412b9bad9194413faeeda0d15b3cdb2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59c38762b52b47b2a417ceb0cee2f0b1 in term 0.
I20260811 05:12:46.251139 31535 raft_consensus.cc:2454] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 36cb15aca8b54ba9b7509260d9483543 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59c38762b52b47b2a417ceb0cee2f0b1 in term 0.
I20260811 05:12:46.251368 31670 leader_election.cc:302] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 59c38762b52b47b2a417ceb0cee2f0b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 412b9bad9194413faeeda0d15b3cdb2d, 59c38762b52b47b2a417ceb0cee2f0b1; no voters:
I20260811 05:12:46.251502 32176 raft_consensus.cc:2781] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 59c38762b52b47b2a417ceb0cee2f0b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:12:46.251564 32176 raft_consensus.cc:482] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 59c38762b52b47b2a417ceb0cee2f0b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:12:46.251590 32176 raft_consensus.cc:3037] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 59c38762b52b47b2a417ceb0cee2f0b1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:46.252519 32176 raft_consensus.cc:504] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 59c38762b52b47b2a417ceb0cee2f0b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36cb15aca8b54ba9b7509260d9483543" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36521 } } peers { permanent_uuid: "412b9bad9194413faeeda0d15b3cdb2d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44771 } } peers { permanent_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41625 } }
I20260811 05:12:46.252662 32176 leader_election.cc:288] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 59c38762b52b47b2a417ceb0cee2f0b1 [CANDIDATE]: Term 1 election: Requested vote from peers 36cb15aca8b54ba9b7509260d9483543 (127.28.143.129:36521), 412b9bad9194413faeeda0d15b3cdb2d (127.28.143.131:44771)
I20260811 05:12:46.252835 32034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b48a97f6c6d422dab1a9ee2545bbbd5" candidate_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "412b9bad9194413faeeda0d15b3cdb2d"
I20260811 05:12:46.252827 31535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b48a97f6c6d422dab1a9ee2545bbbd5" candidate_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36cb15aca8b54ba9b7509260d9483543"
I20260811 05:12:46.252907 32034 raft_consensus.cc:3037] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 412b9bad9194413faeeda0d15b3cdb2d [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:46.252909 31535 raft_consensus.cc:3037] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 36cb15aca8b54ba9b7509260d9483543 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:46.253598 32034 raft_consensus.cc:2454] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 412b9bad9194413faeeda0d15b3cdb2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59c38762b52b47b2a417ceb0cee2f0b1 in term 1.
I20260811 05:12:46.253618 31535 raft_consensus.cc:2454] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 36cb15aca8b54ba9b7509260d9483543 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59c38762b52b47b2a417ceb0cee2f0b1 in term 1.
I20260811 05:12:46.253762 31670 leader_election.cc:302] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 59c38762b52b47b2a417ceb0cee2f0b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36cb15aca8b54ba9b7509260d9483543, 59c38762b52b47b2a417ceb0cee2f0b1; no voters:
I20260811 05:12:46.253859 32176 raft_consensus.cc:2781] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 59c38762b52b47b2a417ceb0cee2f0b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:46.254030 32176 raft_consensus.cc:686] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 59c38762b52b47b2a417ceb0cee2f0b1 [term 1 LEADER]: Becoming Leader. State: Replica: 59c38762b52b47b2a417ceb0cee2f0b1, State: Running, Role: LEADER
I20260811 05:12:46.254135 32176 consensus_queue.cc:237] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 59c38762b52b47b2a417ceb0cee2f0b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36cb15aca8b54ba9b7509260d9483543" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36521 } } peers { permanent_uuid: "412b9bad9194413faeeda0d15b3cdb2d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44771 } } peers { permanent_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41625 } }
I20260811 05:12:46.254977 31226 catalog_manager.cc:5495] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 59c38762b52b47b2a417ceb0cee2f0b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 59c38762b52b47b2a417ceb0cee2f0b1 (127.28.143.130). New cstate: current_term: 1 leader_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36cb15aca8b54ba9b7509260d9483543" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "412b9bad9194413faeeda0d15b3cdb2d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41625 } health_report { overall_health: HEALTHY } } }
I20260811 05:12:46.283118 32155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:46.280457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41024 (local address 127.28.143.130:41625)
0811 05:12:46.280519 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:46.280522 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:46.280566 (+ 44us) server_negotiation.cc:407] Connection header received
0811 05:12:46.280701 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:46.280701 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:46.280724 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:46.280779 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:12:46.280942 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:46.281411 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:46.282249 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:46.282449 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:46.282505 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:46.282720 (+ 215us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:12:46.282835 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:12:46.282945 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:46.282965 (+ 20us) server_negotiation.cc:299] Negotiation successful
0811 05:12:46.282992 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 05:12:46.285367 32034 raft_consensus.cc:1261] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 412b9bad9194413faeeda0d15b3cdb2d [term 1 FOLLOWER]: Refusing update from remote peer 59c38762b52b47b2a417ceb0cee2f0b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:12:46.285751 32181 consensus_queue.cc:1035] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 59c38762b52b47b2a417ceb0cee2f0b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "412b9bad9194413faeeda0d15b3cdb2d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:12:46.286852 31535 raft_consensus.cc:1261] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 36cb15aca8b54ba9b7509260d9483543 [term 1 FOLLOWER]: Refusing update from remote peer 59c38762b52b47b2a417ceb0cee2f0b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:12:46.287163 32176 consensus_queue.cc:1035] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 59c38762b52b47b2a417ceb0cee2f0b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36cb15aca8b54ba9b7509260d9483543" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:12:46.290017 32198 mvcc.cc:205] Tried to move back new op lower bound from 7317197481101234176 to 7317197480977399808. Current Snapshot: MvccSnapshot[applied={T|T < 7317197481101234176}]
I20260811 05:12:46.294184 31228 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54838:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:12:46.294394 31228 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54838:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:12:46.295636 31228 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8798dcd480634809ac8bfa3f15e20049: Sending DeleteTablet for 3 replicas of tablet 4b48a97f6c6d422dab1a9ee2545bbbd5
I20260811 05:12:46.296247 31761 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b48a97f6c6d422dab1a9ee2545bbbd5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:12:46 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:41018
I20260811 05:12:46.296254 32002 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b48a97f6c6d422dab1a9ee2545bbbd5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:12:46 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:49588
I20260811 05:12:46.296702 31501 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b48a97f6c6d422dab1a9ee2545bbbd5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:12:46 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:56450
I20260811 05:12:46.296984 32214 tablet_replica.cc:330] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 412b9bad9194413faeeda0d15b3cdb2d: stopping tablet replica
I20260811 05:12:46.297039 31228 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54838:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 05:12:46.297135 32214 raft_consensus.cc:2229] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 412b9bad9194413faeeda0d15b3cdb2d [term 1 FOLLOWER]: Raft consensus shutting down.
W20260811 05:12:46.297223 31228 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:12:46.297273 32214 raft_consensus.cc:2258] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 412b9bad9194413faeeda0d15b3cdb2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:12:46.297461 32207 tablet_replica.cc:330] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 59c38762b52b47b2a417ceb0cee2f0b1: stopping tablet replica
I20260811 05:12:46.297565 32207 raft_consensus.cc:2229] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 59c38762b52b47b2a417ceb0cee2f0b1 [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:12:46.298064 32214 ts_tablet_manager.cc:1905] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 412b9bad9194413faeeda0d15b3cdb2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:12:46.298852 32002 tablet_service.cc:1467] Processing CreateTablet for tablet 694e820b5da34a3d926e9902099aed92 (DEFAULT_TABLE table=test-table [id=d699fe7d87e6413ebe85f1a2dce1a9f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:12:46.298843 31501 tablet_service.cc:1467] Processing CreateTablet for tablet 694e820b5da34a3d926e9902099aed92 (DEFAULT_TABLE table=test-table [id=d699fe7d87e6413ebe85f1a2dce1a9f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:12:46.299002 32002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 694e820b5da34a3d926e9902099aed92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:46.299007 31501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 694e820b5da34a3d926e9902099aed92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:46.300469 32169 tablet_bootstrap.cc:492] T 694e820b5da34a3d926e9902099aed92 P 36cb15aca8b54ba9b7509260d9483543: Bootstrap starting.
I20260811 05:12:46.300554 32158 tablet_bootstrap.cc:492] T 694e820b5da34a3d926e9902099aed92 P 412b9bad9194413faeeda0d15b3cdb2d: Bootstrap starting.
I20260811 05:12:46.300915 32169 tablet_bootstrap.cc:654] T 694e820b5da34a3d926e9902099aed92 P 36cb15aca8b54ba9b7509260d9483543: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:46.301213 32158 tablet_bootstrap.cc:654] T 694e820b5da34a3d926e9902099aed92 P 412b9bad9194413faeeda0d15b3cdb2d: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:46.301537 32169 tablet_bootstrap.cc:492] T 694e820b5da34a3d926e9902099aed92 P 36cb15aca8b54ba9b7509260d9483543: No bootstrap required, opened a new log
I20260811 05:12:46.301589 32169 ts_tablet_manager.cc:1397] T 694e820b5da34a3d926e9902099aed92 P 36cb15aca8b54ba9b7509260d9483543: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:12:46.301748 32169 raft_consensus.cc:348] T 694e820b5da34a3d926e9902099aed92 P 36cb15aca8b54ba9b7509260d9483543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36cb15aca8b54ba9b7509260d9483543" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36521 } } peers { permanent_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41625 } } peers { permanent_uuid: "412b9bad9194413faeeda0d15b3cdb2d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44771 } }
I20260811 05:12:46.301812 32169 raft_consensus.cc:374] T 694e820b5da34a3d926e9902099aed92 P 36cb15aca8b54ba9b7509260d9483543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:46.301826 32169 raft_consensus.cc:729] T 694e820b5da34a3d926e9902099aed92 P 36cb15aca8b54ba9b7509260d9483543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36cb15aca8b54ba9b7509260d9483543, State: Initialized, Role: FOLLOWER
I20260811 05:12:46.301863 32169 consensus_queue.cc:260] T 694e820b5da34a3d926e9902099aed92 P 36cb15aca8b54ba9b7509260d9483543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36cb15aca8b54ba9b7509260d9483543" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36521 } } peers { permanent_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41625 } } peers { permanent_uuid: "412b9bad9194413faeeda0d15b3cdb2d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44771 } }
I20260811 05:12:46.301970 32169 ts_tablet_manager.cc:1428] T 694e820b5da34a3d926e9902099aed92 P 36cb15aca8b54ba9b7509260d9483543: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:46.301987 32158 tablet_bootstrap.cc:492] T 694e820b5da34a3d926e9902099aed92 P 412b9bad9194413faeeda0d15b3cdb2d: No bootstrap required, opened a new log
I20260811 05:12:46.302031 32158 ts_tablet_manager.cc:1397] T 694e820b5da34a3d926e9902099aed92 P 412b9bad9194413faeeda0d15b3cdb2d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:12:46.302177 32158 raft_consensus.cc:348] T 694e820b5da34a3d926e9902099aed92 P 412b9bad9194413faeeda0d15b3cdb2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36cb15aca8b54ba9b7509260d9483543" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36521 } } peers { permanent_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41625 } } peers { permanent_uuid: "412b9bad9194413faeeda0d15b3cdb2d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44771 } }
I20260811 05:12:46.302259 32158 raft_consensus.cc:374] T 694e820b5da34a3d926e9902099aed92 P 412b9bad9194413faeeda0d15b3cdb2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:46.302286 32158 raft_consensus.cc:729] T 694e820b5da34a3d926e9902099aed92 P 412b9bad9194413faeeda0d15b3cdb2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 412b9bad9194413faeeda0d15b3cdb2d, State: Initialized, Role: FOLLOWER
I20260811 05:12:46.302333 32158 consensus_queue.cc:260] T 694e820b5da34a3d926e9902099aed92 P 412b9bad9194413faeeda0d15b3cdb2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36cb15aca8b54ba9b7509260d9483543" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36521 } } peers { permanent_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41625 } } peers { permanent_uuid: "412b9bad9194413faeeda0d15b3cdb2d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44771 } }
I20260811 05:12:46.302431 32158 ts_tablet_manager.cc:1428] T 694e820b5da34a3d926e9902099aed92 P 412b9bad9194413faeeda0d15b3cdb2d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:46.301695 31761 tablet_service.cc:1467] Processing CreateTablet for tablet 694e820b5da34a3d926e9902099aed92 (DEFAULT_TABLE table=test-table [id=d699fe7d87e6413ebe85f1a2dce1a9f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:12:46.303095 31761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 694e820b5da34a3d926e9902099aed92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:46.304574 32160 tablet_bootstrap.cc:492] T 694e820b5da34a3d926e9902099aed92 P 59c38762b52b47b2a417ceb0cee2f0b1: Bootstrap starting.
I20260811 05:12:46.304876 32207 raft_consensus.cc:2258] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 59c38762b52b47b2a417ceb0cee2f0b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:12:46.305064 32160 tablet_bootstrap.cc:654] T 694e820b5da34a3d926e9902099aed92 P 59c38762b52b47b2a417ceb0cee2f0b1: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:46.305410 32207 ts_tablet_manager.cc:1905] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 59c38762b52b47b2a417ceb0cee2f0b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:12:46.305693 32160 tablet_bootstrap.cc:492] T 694e820b5da34a3d926e9902099aed92 P 59c38762b52b47b2a417ceb0cee2f0b1: No bootstrap required, opened a new log
I20260811 05:12:46.305742 32160 ts_tablet_manager.cc:1397] T 694e820b5da34a3d926e9902099aed92 P 59c38762b52b47b2a417ceb0cee2f0b1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:12:46.305892 32160 raft_consensus.cc:348] T 694e820b5da34a3d926e9902099aed92 P 59c38762b52b47b2a417ceb0cee2f0b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36cb15aca8b54ba9b7509260d9483543" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36521 } } peers { permanent_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41625 } } peers { permanent_uuid: "412b9bad9194413faeeda0d15b3cdb2d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44771 } }
I20260811 05:12:46.305930 32214 ts_tablet_manager.cc:1918] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 412b9bad9194413faeeda0d15b3cdb2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:12:46.305968 32160 raft_consensus.cc:374] T 694e820b5da34a3d926e9902099aed92 P 59c38762b52b47b2a417ceb0cee2f0b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:46.305987 32160 raft_consensus.cc:729] T 694e820b5da34a3d926e9902099aed92 P 59c38762b52b47b2a417ceb0cee2f0b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59c38762b52b47b2a417ceb0cee2f0b1, State: Initialized, Role: FOLLOWER
I20260811 05:12:46.306020 32214 log.cc:1196] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 412b9bad9194413faeeda0d15b3cdb2d: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-2/wal/wals/4b48a97f6c6d422dab1a9ee2545bbbd5
I20260811 05:12:46.306030 32160 consensus_queue.cc:260] T 694e820b5da34a3d926e9902099aed92 P 59c38762b52b47b2a417ceb0cee2f0b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36cb15aca8b54ba9b7509260d9483543" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36521 } } peers { permanent_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41625 } } peers { permanent_uuid: "412b9bad9194413faeeda0d15b3cdb2d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44771 } }
I20260811 05:12:46.306123 32160 ts_tablet_manager.cc:1428] T 694e820b5da34a3d926e9902099aed92 P 59c38762b52b47b2a417ceb0cee2f0b1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:46.306334 32214 ts_tablet_manager.cc:1939] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 412b9bad9194413faeeda0d15b3cdb2d: Deleting consensus metadata
I20260811 05:12:46.306326 32216 tablet_replica.cc:330] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 36cb15aca8b54ba9b7509260d9483543: stopping tablet replica
I20260811 05:12:46.306468 32216 raft_consensus.cc:2229] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 36cb15aca8b54ba9b7509260d9483543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:12:46.306591 32216 raft_consensus.cc:2258] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 36cb15aca8b54ba9b7509260d9483543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:12:46.306788 31199 catalog_manager.cc:4841] TS 412b9bad9194413faeeda0d15b3cdb2d (127.28.143.131:44771): tablet 4b48a97f6c6d422dab1a9ee2545bbbd5 (table test-table [id=48d6d2de2b3b49308b5d87f03ed9e91e]) successfully deleted
I20260811 05:12:46.307053 32207 ts_tablet_manager.cc:1918] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 59c38762b52b47b2a417ceb0cee2f0b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:12:46.307130 32207 log.cc:1196] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 59c38762b52b47b2a417ceb0cee2f0b1: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-1/wal/wals/4b48a97f6c6d422dab1a9ee2545bbbd5
I20260811 05:12:46.307646 32207 ts_tablet_manager.cc:1939] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 59c38762b52b47b2a417ceb0cee2f0b1: Deleting consensus metadata
I20260811 05:12:46.307768 32216 ts_tablet_manager.cc:1905] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 36cb15aca8b54ba9b7509260d9483543: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:12:46.308070 31204 catalog_manager.cc:4841] TS 59c38762b52b47b2a417ceb0cee2f0b1 (127.28.143.130:41625): tablet 4b48a97f6c6d422dab1a9ee2545bbbd5 (table test-table [id=48d6d2de2b3b49308b5d87f03ed9e91e]) successfully deleted
I20260811 05:12:46.309784 32216 ts_tablet_manager.cc:1918] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 36cb15aca8b54ba9b7509260d9483543: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:12:46.309880 32216 log.cc:1196] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 36cb15aca8b54ba9b7509260d9483543: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-0/wal/wals/4b48a97f6c6d422dab1a9ee2545bbbd5
I20260811 05:12:46.310140 32216 ts_tablet_manager.cc:1939] T 4b48a97f6c6d422dab1a9ee2545bbbd5 P 36cb15aca8b54ba9b7509260d9483543: Deleting consensus metadata
I20260811 05:12:46.310570 31199 catalog_manager.cc:4841] TS 36cb15aca8b54ba9b7509260d9483543 (127.28.143.129:36521): tablet 4b48a97f6c6d422dab1a9ee2545bbbd5 (table test-table [id=48d6d2de2b3b49308b5d87f03ed9e91e]) successfully deleted
I20260811 05:12:46.403447 32177 raft_consensus.cc:482] T 694e820b5da34a3d926e9902099aed92 P 412b9bad9194413faeeda0d15b3cdb2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:12:46.403569 32177 raft_consensus.cc:504] T 694e820b5da34a3d926e9902099aed92 P 412b9bad9194413faeeda0d15b3cdb2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36cb15aca8b54ba9b7509260d9483543" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36521 } } peers { permanent_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41625 } } peers { permanent_uuid: "412b9bad9194413faeeda0d15b3cdb2d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44771 } }
I20260811 05:12:46.403947 32177 leader_election.cc:288] T 694e820b5da34a3d926e9902099aed92 P 412b9bad9194413faeeda0d15b3cdb2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 36cb15aca8b54ba9b7509260d9483543 (127.28.143.129:36521), 59c38762b52b47b2a417ceb0cee2f0b1 (127.28.143.130:41625)
I20260811 05:12:46.407680 31925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:46.404155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:36521 (local address 127.28.143.131:47209)
0811 05:12:46.404242 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:46.404255 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:46.404305 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:46.404500 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:46.404502 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:46.404503 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:46.404582 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:46.404584 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:46.405608 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:46.405609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:46.406792 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:46.406796 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:46.407399 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:46.407401 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:46.407477 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:46.407481 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:12:46.407526 (+ 45us) client_negotiation.cc:240] Negotiation successful
0811 05:12:46.407573 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65,"mutex_wait_us":51}
I20260811 05:12:46.407665 32153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:46.404236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:47209 (local address 127.28.143.129:36521)
0811 05:12:46.404282 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:46.404284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:46.404294 (+ 10us) server_negotiation.cc:407] Connection header received
0811 05:12:46.404392 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:46.404393 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:46.404404 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:46.404450 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:46.404747 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:46.405507 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:46.406947 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:46.407313 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:46.407381 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:46.407393 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:46.407552 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 05:12:46.407597 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 05:12:46.407956 32155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:46.404400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:43409 (local address 127.28.143.130:41625)
0811 05:12:46.404447 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:46.404450 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:46.404461 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:12:46.404512 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:46.404513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:46.404524 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:46.404575 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:46.404770 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:46.405274 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:46.406942 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:46.407539 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:46.407619 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:46.407628 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:46.407842 (+ 214us) server_negotiation.cc:299] Negotiation successful
0811 05:12:46.407883 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 05:12:46.408025 31535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "694e820b5da34a3d926e9902099aed92" candidate_uuid: "412b9bad9194413faeeda0d15b3cdb2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36cb15aca8b54ba9b7509260d9483543" is_pre_election: true
I20260811 05:12:46.408130 31535 raft_consensus.cc:2454] T 694e820b5da34a3d926e9902099aed92 P 36cb15aca8b54ba9b7509260d9483543 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 412b9bad9194413faeeda0d15b3cdb2d in term 0.
I20260811 05:12:46.408231 32224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:46.404194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:41625 (local address 127.28.143.131:43409)
0811 05:12:46.404343 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:46.404353 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:46.404441 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:46.404631 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:46.404632 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:46.404634 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:46.404705 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:46.404707 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:46.405385 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:46.405386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:46.406792 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:46.406796 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:46.407686 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:46.407687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:46.407764 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:46.407769 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 05:12:46.408082 (+ 313us) client_negotiation.cc:240] Negotiation successful
0811 05:12:46.408126 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":1,"thread_start_us":71,"threads_started":1}
I20260811 05:12:46.408336 31914 leader_election.cc:302] T 694e820b5da34a3d926e9902099aed92 P 412b9bad9194413faeeda0d15b3cdb2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36cb15aca8b54ba9b7509260d9483543, 412b9bad9194413faeeda0d15b3cdb2d; no voters:
I20260811 05:12:46.408488 32177 raft_consensus.cc:2781] T 694e820b5da34a3d926e9902099aed92 P 412b9bad9194413faeeda0d15b3cdb2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:12:46.408521 32177 raft_consensus.cc:482] T 694e820b5da34a3d926e9902099aed92 P 412b9bad9194413faeeda0d15b3cdb2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:12:46.408541 32177 raft_consensus.cc:3037] T 694e820b5da34a3d926e9902099aed92 P 412b9bad9194413faeeda0d15b3cdb2d [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:46.408795 31791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "694e820b5da34a3d926e9902099aed92" candidate_uuid: "412b9bad9194413faeeda0d15b3cdb2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" is_pre_election: true
I20260811 05:12:46.408918 31791 raft_consensus.cc:2454] T 694e820b5da34a3d926e9902099aed92 P 59c38762b52b47b2a417ceb0cee2f0b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 412b9bad9194413faeeda0d15b3cdb2d in term 0.
I20260811 05:12:46.409489 32177 raft_consensus.cc:504] T 694e820b5da34a3d926e9902099aed92 P 412b9bad9194413faeeda0d15b3cdb2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36cb15aca8b54ba9b7509260d9483543" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36521 } } peers { permanent_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41625 } } peers { permanent_uuid: "412b9bad9194413faeeda0d15b3cdb2d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44771 } }
I20260811 05:12:46.409634 32177 leader_election.cc:288] T 694e820b5da34a3d926e9902099aed92 P 412b9bad9194413faeeda0d15b3cdb2d [CANDIDATE]: Term 1 election: Requested vote from peers 36cb15aca8b54ba9b7509260d9483543 (127.28.143.129:36521), 59c38762b52b47b2a417ceb0cee2f0b1 (127.28.143.130:41625)
I20260811 05:12:46.409862 31535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "694e820b5da34a3d926e9902099aed92" candidate_uuid: "412b9bad9194413faeeda0d15b3cdb2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36cb15aca8b54ba9b7509260d9483543"
I20260811 05:12:46.409937 31535 raft_consensus.cc:3037] T 694e820b5da34a3d926e9902099aed92 P 36cb15aca8b54ba9b7509260d9483543 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:46.410583 31535 raft_consensus.cc:2454] T 694e820b5da34a3d926e9902099aed92 P 36cb15aca8b54ba9b7509260d9483543 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 412b9bad9194413faeeda0d15b3cdb2d in term 1.
I20260811 05:12:46.410718 31791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "694e820b5da34a3d926e9902099aed92" candidate_uuid: "412b9bad9194413faeeda0d15b3cdb2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59c38762b52b47b2a417ceb0cee2f0b1"
I20260811 05:12:46.410768 31914 leader_election.cc:302] T 694e820b5da34a3d926e9902099aed92 P 412b9bad9194413faeeda0d15b3cdb2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36cb15aca8b54ba9b7509260d9483543, 412b9bad9194413faeeda0d15b3cdb2d; no voters:
I20260811 05:12:46.410804 31791 raft_consensus.cc:3037] T 694e820b5da34a3d926e9902099aed92 P 59c38762b52b47b2a417ceb0cee2f0b1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:46.410845 32177 raft_consensus.cc:2781] T 694e820b5da34a3d926e9902099aed92 P 412b9bad9194413faeeda0d15b3cdb2d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:46.410988 32177 raft_consensus.cc:686] T 694e820b5da34a3d926e9902099aed92 P 412b9bad9194413faeeda0d15b3cdb2d [term 1 LEADER]: Becoming Leader. State: Replica: 412b9bad9194413faeeda0d15b3cdb2d, State: Running, Role: LEADER
I20260811 05:12:46.411067 32177 consensus_queue.cc:237] T 694e820b5da34a3d926e9902099aed92 P 412b9bad9194413faeeda0d15b3cdb2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36cb15aca8b54ba9b7509260d9483543" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36521 } } peers { permanent_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41625 } } peers { permanent_uuid: "412b9bad9194413faeeda0d15b3cdb2d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44771 } }
I20260811 05:12:46.411676 31791 raft_consensus.cc:2454] T 694e820b5da34a3d926e9902099aed92 P 59c38762b52b47b2a417ceb0cee2f0b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 412b9bad9194413faeeda0d15b3cdb2d in term 1.
I20260811 05:12:46.411875 31224 catalog_manager.cc:5495] T 694e820b5da34a3d926e9902099aed92 P 412b9bad9194413faeeda0d15b3cdb2d reported cstate change: term changed from 0 to 1, leader changed from <none> to 412b9bad9194413faeeda0d15b3cdb2d (127.28.143.131). New cstate: current_term: 1 leader_uuid: "412b9bad9194413faeeda0d15b3cdb2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36cb15aca8b54ba9b7509260d9483543" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "412b9bad9194413faeeda0d15b3cdb2d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44771 } health_report { overall_health: HEALTHY } } }
Aug 11 05:12:46 dist-test-slave-f7th krb5kdc[31141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425165, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:12:46.453558 32241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:46.445607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:34053 (local address 127.0.0.1:54852)
0811 05:12:46.445792 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:46.445802 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:46.445873 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:46.446096 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:46.446098 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:46.446400 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:46.446447 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:46.446449 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:46.447188 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:46.447189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:46.447740 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:46.447742 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:46.448163 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:46.448164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:46.448212 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:46.448792 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:12:46.448797 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:46.450362 (+ 1565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:46.452253 (+ 1891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:46.452263 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:46.452265 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:46.452504 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:46.452783 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:46.452785 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:46.452785 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:46.452855 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:46.453225 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:46.453234 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:46.453289 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 05:12:46.453357 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 05:12:46.453478 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":98,"threads_started":1}
I20260811 05:12:46.453629 31351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:46.445716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54852 (local address 127.28.143.190:34053)
0811 05:12:46.445810 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:46.445812 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:46.445917 (+ 105us) server_negotiation.cc:407] Connection header received
0811 05:12:46.445943 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:46.445943 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:46.445953 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:46.445994 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:46.446530 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:46.447070 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:46.447839 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:46.448079 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:46.448144 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:46.450472 (+ 2328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:46.450474 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:46.450474 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:46.450482 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:46.452153 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:46.452582 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:46.452583 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:46.452583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:46.452700 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:46.452984 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:46.452985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:46.452985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:46.453133 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:46.453199 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:46.453378 (+ 179us) server_negotiation.cc:299] Negotiation successful
0811 05:12:46.453544 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":79,"server-negotiator.queue_time_us":66}
I20260811 05:12:46.456188 31224 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:54852:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 05:12:46.456498 31224 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:12:46.458225 31501 tablet_service.cc:1467] Processing CreateTablet for tablet ebefde6c00494c1899b4bf0f590b5d6c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5a9ba13ca15142f18a7a629129464444]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 05:12:46.458176 32002 tablet_service.cc:1467] Processing CreateTablet for tablet ebefde6c00494c1899b4bf0f590b5d6c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5a9ba13ca15142f18a7a629129464444]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 05:12:46.458212 31761 tablet_service.cc:1467] Processing CreateTablet for tablet ebefde6c00494c1899b4bf0f590b5d6c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5a9ba13ca15142f18a7a629129464444]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 05:12:46.458443 31501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebefde6c00494c1899b4bf0f590b5d6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:46.458443 32002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebefde6c00494c1899b4bf0f590b5d6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:46.458562 31761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebefde6c00494c1899b4bf0f590b5d6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:46.460007 32160 tablet_bootstrap.cc:492] T ebefde6c00494c1899b4bf0f590b5d6c P 59c38762b52b47b2a417ceb0cee2f0b1: Bootstrap starting.
I20260811 05:12:46.460043 32158 tablet_bootstrap.cc:492] T ebefde6c00494c1899b4bf0f590b5d6c P 412b9bad9194413faeeda0d15b3cdb2d: Bootstrap starting.
I20260811 05:12:46.460150 32169 tablet_bootstrap.cc:492] T ebefde6c00494c1899b4bf0f590b5d6c P 36cb15aca8b54ba9b7509260d9483543: Bootstrap starting.
I20260811 05:12:46.460445 32158 tablet_bootstrap.cc:654] T ebefde6c00494c1899b4bf0f590b5d6c P 412b9bad9194413faeeda0d15b3cdb2d: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:46.460445 32160 tablet_bootstrap.cc:654] T ebefde6c00494c1899b4bf0f590b5d6c P 59c38762b52b47b2a417ceb0cee2f0b1: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:46.460532 32169 tablet_bootstrap.cc:654] T ebefde6c00494c1899b4bf0f590b5d6c P 36cb15aca8b54ba9b7509260d9483543: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:46.461341 32169 tablet_bootstrap.cc:492] T ebefde6c00494c1899b4bf0f590b5d6c P 36cb15aca8b54ba9b7509260d9483543: No bootstrap required, opened a new log
I20260811 05:12:46.461341 32158 tablet_bootstrap.cc:492] T ebefde6c00494c1899b4bf0f590b5d6c P 412b9bad9194413faeeda0d15b3cdb2d: No bootstrap required, opened a new log
I20260811 05:12:46.461414 32158 ts_tablet_manager.cc:1397] T ebefde6c00494c1899b4bf0f590b5d6c P 412b9bad9194413faeeda0d15b3cdb2d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:12:46.461414 32169 ts_tablet_manager.cc:1397] T ebefde6c00494c1899b4bf0f590b5d6c P 36cb15aca8b54ba9b7509260d9483543: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:12:46.461580 32158 raft_consensus.cc:348] T ebefde6c00494c1899b4bf0f590b5d6c P 412b9bad9194413faeeda0d15b3cdb2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41625 } } peers { permanent_uuid: "412b9bad9194413faeeda0d15b3cdb2d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44771 } } peers { permanent_uuid: "36cb15aca8b54ba9b7509260d9483543" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36521 } }
I20260811 05:12:46.461580 32169 raft_consensus.cc:348] T ebefde6c00494c1899b4bf0f590b5d6c P 36cb15aca8b54ba9b7509260d9483543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41625 } } peers { permanent_uuid: "412b9bad9194413faeeda0d15b3cdb2d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44771 } } peers { permanent_uuid: "36cb15aca8b54ba9b7509260d9483543" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36521 } }
I20260811 05:12:46.461661 32169 raft_consensus.cc:374] T ebefde6c00494c1899b4bf0f590b5d6c P 36cb15aca8b54ba9b7509260d9483543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:46.461661 32158 raft_consensus.cc:374] T ebefde6c00494c1899b4bf0f590b5d6c P 412b9bad9194413faeeda0d15b3cdb2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:46.461683 32158 raft_consensus.cc:729] T ebefde6c00494c1899b4bf0f590b5d6c P 412b9bad9194413faeeda0d15b3cdb2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 412b9bad9194413faeeda0d15b3cdb2d, State: Initialized, Role: FOLLOWER
I20260811 05:12:46.461683 32169 raft_consensus.cc:729] T ebefde6c00494c1899b4bf0f590b5d6c P 36cb15aca8b54ba9b7509260d9483543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36cb15aca8b54ba9b7509260d9483543, State: Initialized, Role: FOLLOWER
I20260811 05:12:46.461735 32169 consensus_queue.cc:260] T ebefde6c00494c1899b4bf0f590b5d6c P 36cb15aca8b54ba9b7509260d9483543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41625 } } peers { permanent_uuid: "412b9bad9194413faeeda0d15b3cdb2d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44771 } } peers { permanent_uuid: "36cb15aca8b54ba9b7509260d9483543" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36521 } }
I20260811 05:12:46.461733 32158 consensus_queue.cc:260] T ebefde6c00494c1899b4bf0f590b5d6c P 412b9bad9194413faeeda0d15b3cdb2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41625 } } peers { permanent_uuid: "412b9bad9194413faeeda0d15b3cdb2d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44771 } } peers { permanent_uuid: "36cb15aca8b54ba9b7509260d9483543" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36521 } }
I20260811 05:12:46.461833 32158 ts_tablet_manager.cc:1428] T ebefde6c00494c1899b4bf0f590b5d6c P 412b9bad9194413faeeda0d15b3cdb2d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 05:12:46.461843 32169 ts_tablet_manager.cc:1428] T ebefde6c00494c1899b4bf0f590b5d6c P 36cb15aca8b54ba9b7509260d9483543: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 05:12:46.461931 32230 tablet_replica.cc:439] T ebefde6c00494c1899b4bf0f590b5d6c P 412b9bad9194413faeeda0d15b3cdb2d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41625 } } peers { permanent_uuid: "412b9bad9194413faeeda0d15b3cdb2d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44771 } } peers { permanent_uuid: "36cb15aca8b54ba9b7509260d9483543" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36521 } } }
I20260811 05:12:46.462060 32230 tablet_replica.cc:442] T ebefde6c00494c1899b4bf0f590b5d6c P 412b9bad9194413faeeda0d15b3cdb2d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:12:46.462174 32160 tablet_bootstrap.cc:492] T ebefde6c00494c1899b4bf0f590b5d6c P 59c38762b52b47b2a417ceb0cee2f0b1: No bootstrap required, opened a new log
I20260811 05:12:46.462252 32160 ts_tablet_manager.cc:1397] T ebefde6c00494c1899b4bf0f590b5d6c P 59c38762b52b47b2a417ceb0cee2f0b1: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 05:12:46.462252 32178 tablet_replica.cc:439] T ebefde6c00494c1899b4bf0f590b5d6c P 36cb15aca8b54ba9b7509260d9483543: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41625 } } peers { permanent_uuid: "412b9bad9194413faeeda0d15b3cdb2d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44771 } } peers { permanent_uuid: "36cb15aca8b54ba9b7509260d9483543" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36521 } } }
I20260811 05:12:46.462342 32178 tablet_replica.cc:442] T ebefde6c00494c1899b4bf0f590b5d6c P 36cb15aca8b54ba9b7509260d9483543: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:12:46.462370 32160 raft_consensus.cc:348] T ebefde6c00494c1899b4bf0f590b5d6c P 59c38762b52b47b2a417ceb0cee2f0b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41625 } } peers { permanent_uuid: "412b9bad9194413faeeda0d15b3cdb2d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44771 } } peers { permanent_uuid: "36cb15aca8b54ba9b7509260d9483543" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36521 } }
I20260811 05:12:46.462433 32160 raft_consensus.cc:374] T ebefde6c00494c1899b4bf0f590b5d6c P 59c38762b52b47b2a417ceb0cee2f0b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:46.462448 32160 raft_consensus.cc:729] T ebefde6c00494c1899b4bf0f590b5d6c P 59c38762b52b47b2a417ceb0cee2f0b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59c38762b52b47b2a417ceb0cee2f0b1, State: Initialized, Role: FOLLOWER
I20260811 05:12:46.462496 32160 consensus_queue.cc:260] T ebefde6c00494c1899b4bf0f590b5d6c P 59c38762b52b47b2a417ceb0cee2f0b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41625 } } peers { permanent_uuid: "412b9bad9194413faeeda0d15b3cdb2d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44771 } } peers { permanent_uuid: "36cb15aca8b54ba9b7509260d9483543" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36521 } }
I20260811 05:12:46.462589 32160 ts_tablet_manager.cc:1428] T ebefde6c00494c1899b4bf0f590b5d6c P 59c38762b52b47b2a417ceb0cee2f0b1: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260811 05:12:46.462695 32181 tablet_replica.cc:439] T ebefde6c00494c1899b4bf0f590b5d6c P 59c38762b52b47b2a417ceb0cee2f0b1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41625 } } peers { permanent_uuid: "412b9bad9194413faeeda0d15b3cdb2d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44771 } } peers { permanent_uuid: "36cb15aca8b54ba9b7509260d9483543" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36521 } } }
I20260811 05:12:46.462769 32181 tablet_replica.cc:442] T ebefde6c00494c1899b4bf0f590b5d6c P 59c38762b52b47b2a417ceb0cee2f0b1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:12:46.626278 32181 raft_consensus.cc:482] T ebefde6c00494c1899b4bf0f590b5d6c P 59c38762b52b47b2a417ceb0cee2f0b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:12:46.626412 32181 raft_consensus.cc:504] T ebefde6c00494c1899b4bf0f590b5d6c P 59c38762b52b47b2a417ceb0cee2f0b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41625 } } peers { permanent_uuid: "412b9bad9194413faeeda0d15b3cdb2d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44771 } } peers { permanent_uuid: "36cb15aca8b54ba9b7509260d9483543" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36521 } }
I20260811 05:12:46.626612 32181 leader_election.cc:288] T ebefde6c00494c1899b4bf0f590b5d6c P 59c38762b52b47b2a417ceb0cee2f0b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 412b9bad9194413faeeda0d15b3cdb2d (127.28.143.131:44771), 36cb15aca8b54ba9b7509260d9483543 (127.28.143.129:36521)
I20260811 05:12:46.626947 32034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebefde6c00494c1899b4bf0f590b5d6c" candidate_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "412b9bad9194413faeeda0d15b3cdb2d" is_pre_election: true
I20260811 05:12:46.626945 31535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebefde6c00494c1899b4bf0f590b5d6c" candidate_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36cb15aca8b54ba9b7509260d9483543" is_pre_election: true
I20260811 05:12:46.627059 31535 raft_consensus.cc:2454] T ebefde6c00494c1899b4bf0f590b5d6c P 36cb15aca8b54ba9b7509260d9483543 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59c38762b52b47b2a417ceb0cee2f0b1 in term 0.
I20260811 05:12:46.627058 32034 raft_consensus.cc:2454] T ebefde6c00494c1899b4bf0f590b5d6c P 412b9bad9194413faeeda0d15b3cdb2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59c38762b52b47b2a417ceb0cee2f0b1 in term 0.
I20260811 05:12:46.627228 31670 leader_election.cc:302] T ebefde6c00494c1899b4bf0f590b5d6c P 59c38762b52b47b2a417ceb0cee2f0b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36cb15aca8b54ba9b7509260d9483543, 59c38762b52b47b2a417ceb0cee2f0b1; no voters:
I20260811 05:12:46.627331 32181 raft_consensus.cc:2781] T ebefde6c00494c1899b4bf0f590b5d6c P 59c38762b52b47b2a417ceb0cee2f0b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:12:46.627403 32181 raft_consensus.cc:482] T ebefde6c00494c1899b4bf0f590b5d6c P 59c38762b52b47b2a417ceb0cee2f0b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:12:46.627425 32181 raft_consensus.cc:3037] T ebefde6c00494c1899b4bf0f590b5d6c P 59c38762b52b47b2a417ceb0cee2f0b1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:46.628502 32181 raft_consensus.cc:504] T ebefde6c00494c1899b4bf0f590b5d6c P 59c38762b52b47b2a417ceb0cee2f0b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41625 } } peers { permanent_uuid: "412b9bad9194413faeeda0d15b3cdb2d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44771 } } peers { permanent_uuid: "36cb15aca8b54ba9b7509260d9483543" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36521 } }
I20260811 05:12:46.628703 32181 leader_election.cc:288] T ebefde6c00494c1899b4bf0f590b5d6c P 59c38762b52b47b2a417ceb0cee2f0b1 [CANDIDATE]: Term 1 election: Requested vote from peers 412b9bad9194413faeeda0d15b3cdb2d (127.28.143.131:44771), 36cb15aca8b54ba9b7509260d9483543 (127.28.143.129:36521)
I20260811 05:12:46.628899 32034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebefde6c00494c1899b4bf0f590b5d6c" candidate_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "412b9bad9194413faeeda0d15b3cdb2d"
I20260811 05:12:46.628901 31535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebefde6c00494c1899b4bf0f590b5d6c" candidate_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36cb15aca8b54ba9b7509260d9483543"
I20260811 05:12:46.628993 32034 raft_consensus.cc:3037] T ebefde6c00494c1899b4bf0f590b5d6c P 412b9bad9194413faeeda0d15b3cdb2d [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:46.628996 31535 raft_consensus.cc:3037] T ebefde6c00494c1899b4bf0f590b5d6c P 36cb15aca8b54ba9b7509260d9483543 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:46.629737 31535 raft_consensus.cc:2454] T ebefde6c00494c1899b4bf0f590b5d6c P 36cb15aca8b54ba9b7509260d9483543 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59c38762b52b47b2a417ceb0cee2f0b1 in term 1.
I20260811 05:12:46.629846 32034 raft_consensus.cc:2454] T ebefde6c00494c1899b4bf0f590b5d6c P 412b9bad9194413faeeda0d15b3cdb2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59c38762b52b47b2a417ceb0cee2f0b1 in term 1.
I20260811 05:12:46.629936 31670 leader_election.cc:302] T ebefde6c00494c1899b4bf0f590b5d6c P 59c38762b52b47b2a417ceb0cee2f0b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36cb15aca8b54ba9b7509260d9483543, 59c38762b52b47b2a417ceb0cee2f0b1; no voters:
I20260811 05:12:46.630052 32181 raft_consensus.cc:2781] T ebefde6c00494c1899b4bf0f590b5d6c P 59c38762b52b47b2a417ceb0cee2f0b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:46.630112 32181 raft_consensus.cc:686] T ebefde6c00494c1899b4bf0f590b5d6c P 59c38762b52b47b2a417ceb0cee2f0b1 [term 1 LEADER]: Becoming Leader. State: Replica: 59c38762b52b47b2a417ceb0cee2f0b1, State: Running, Role: LEADER
I20260811 05:12:46.630170 32181 consensus_queue.cc:237] T ebefde6c00494c1899b4bf0f590b5d6c P 59c38762b52b47b2a417ceb0cee2f0b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41625 } } peers { permanent_uuid: "412b9bad9194413faeeda0d15b3cdb2d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44771 } } peers { permanent_uuid: "36cb15aca8b54ba9b7509260d9483543" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36521 } }
I20260811 05:12:46.630463 32176 tablet_replica.cc:439] T ebefde6c00494c1899b4bf0f590b5d6c P 59c38762b52b47b2a417ceb0cee2f0b1: TxnStatusTablet state changed. Reason: New leader 59c38762b52b47b2a417ceb0cee2f0b1. Latest consensus state: current_term: 1 leader_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41625 } } peers { permanent_uuid: "412b9bad9194413faeeda0d15b3cdb2d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44771 } } peers { permanent_uuid: "36cb15aca8b54ba9b7509260d9483543" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36521 } } }
I20260811 05:12:46.630564 32176 tablet_replica.cc:442] T ebefde6c00494c1899b4bf0f590b5d6c P 59c38762b52b47b2a417ceb0cee2f0b1: This TxnStatusTablet replica's current role is: LEADER
I20260811 05:12:46.630728 32247 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 05:12:46.630872 32247 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 05:12:46.630883 31224 catalog_manager.cc:5495] T ebefde6c00494c1899b4bf0f590b5d6c P 59c38762b52b47b2a417ceb0cee2f0b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 59c38762b52b47b2a417ceb0cee2f0b1 (127.28.143.130). New cstate: current_term: 1 leader_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "412b9bad9194413faeeda0d15b3cdb2d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36cb15aca8b54ba9b7509260d9483543" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36521 } health_report { overall_health: UNKNOWN } } }
I20260811 05:12:46.649706 32241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:46.647249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:41625 (local address 127.0.0.1:41036)
0811 05:12:46.647359 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:46.647373 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:46.647453 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:46.647604 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:46.647605 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:46.647606 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 05:12:46.647662 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:46.647664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:46.648278 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:46.648279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:46.648946 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:46.648949 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:46.649288 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:46.649289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:46.649341 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:46.649363 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 05:12:46.649525 (+ 162us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 05:12:46.649534 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 05:12:46.649577 (+ 43us) client_negotiation.cc:240] Negotiation successful
0811 05:12:46.649637 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 05:12:46.649701 32155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:46.647391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41036 (local address 127.28.143.130:41625)
0811 05:12:46.647451 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:46.647453 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:46.647462 (+ 9us) server_negotiation.cc:407] Connection header received
0811 05:12:46.647534 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:46.647535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:46.647546 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:46.647592 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:12:46.647717 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:46.648206 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:46.649037 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:46.649226 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:46.649283 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:46.649439 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:12:46.649476 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:12:46.649519 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:46.649586 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 05:12:46.649613 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 05:12:46.651298 31535 raft_consensus.cc:1261] T ebefde6c00494c1899b4bf0f590b5d6c P 36cb15aca8b54ba9b7509260d9483543 [term 1 FOLLOWER]: Refusing update from remote peer 59c38762b52b47b2a417ceb0cee2f0b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:12:46.651309 32034 raft_consensus.cc:1261] T ebefde6c00494c1899b4bf0f590b5d6c P 412b9bad9194413faeeda0d15b3cdb2d [term 1 FOLLOWER]: Refusing update from remote peer 59c38762b52b47b2a417ceb0cee2f0b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:12:46.651571 32176 consensus_queue.cc:1035] T ebefde6c00494c1899b4bf0f590b5d6c P 59c38762b52b47b2a417ceb0cee2f0b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36cb15aca8b54ba9b7509260d9483543" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:12:46.651687 32181 consensus_queue.cc:1035] T ebefde6c00494c1899b4bf0f590b5d6c P 59c38762b52b47b2a417ceb0cee2f0b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "412b9bad9194413faeeda0d15b3cdb2d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:12:46.652329 32178 tablet_replica.cc:439] T ebefde6c00494c1899b4bf0f590b5d6c P 36cb15aca8b54ba9b7509260d9483543: TxnStatusTablet state changed. Reason: New leader 59c38762b52b47b2a417ceb0cee2f0b1. Latest consensus state: current_term: 1 leader_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41625 } } peers { permanent_uuid: "412b9bad9194413faeeda0d15b3cdb2d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44771 } } peers { permanent_uuid: "36cb15aca8b54ba9b7509260d9483543" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36521 } } }
I20260811 05:12:46.652334 32230 tablet_replica.cc:439] T ebefde6c00494c1899b4bf0f590b5d6c P 412b9bad9194413faeeda0d15b3cdb2d: TxnStatusTablet state changed. Reason: New leader 59c38762b52b47b2a417ceb0cee2f0b1. Latest consensus state: current_term: 1 leader_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41625 } } peers { permanent_uuid: "412b9bad9194413faeeda0d15b3cdb2d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44771 } } peers { permanent_uuid: "36cb15aca8b54ba9b7509260d9483543" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36521 } } }
I20260811 05:12:46.652436 32178 tablet_replica.cc:442] T ebefde6c00494c1899b4bf0f590b5d6c P 36cb15aca8b54ba9b7509260d9483543: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:12:46.652455 32230 tablet_replica.cc:442] T ebefde6c00494c1899b4bf0f590b5d6c P 412b9bad9194413faeeda0d15b3cdb2d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:12:46.653234 32176 tablet_replica.cc:439] T ebefde6c00494c1899b4bf0f590b5d6c P 59c38762b52b47b2a417ceb0cee2f0b1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41625 } } peers { permanent_uuid: "412b9bad9194413faeeda0d15b3cdb2d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44771 } } peers { permanent_uuid: "36cb15aca8b54ba9b7509260d9483543" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36521 } } }
I20260811 05:12:46.653327 32176 tablet_replica.cc:442] T ebefde6c00494c1899b4bf0f590b5d6c P 59c38762b52b47b2a417ceb0cee2f0b1: This TxnStatusTablet replica's current role is: LEADER
I20260811 05:12:46.653501 32187 mvcc.cc:205] Tried to move back new op lower bound from 7317197482601635840 to 7317197482517504000. Current Snapshot: MvccSnapshot[applied={T|T < 7317197482601635840 or (T in {7317197482601635840})}]
I20260811 05:12:46.653658 32178 tablet_replica.cc:439] T ebefde6c00494c1899b4bf0f590b5d6c P 36cb15aca8b54ba9b7509260d9483543: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41625 } } peers { permanent_uuid: "412b9bad9194413faeeda0d15b3cdb2d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44771 } } peers { permanent_uuid: "36cb15aca8b54ba9b7509260d9483543" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36521 } } }
I20260811 05:12:46.653749 32178 tablet_replica.cc:442] T ebefde6c00494c1899b4bf0f590b5d6c P 36cb15aca8b54ba9b7509260d9483543: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:12:46.654592 32176 tablet_replica.cc:439] T ebefde6c00494c1899b4bf0f590b5d6c P 59c38762b52b47b2a417ceb0cee2f0b1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41625 } } peers { permanent_uuid: "412b9bad9194413faeeda0d15b3cdb2d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44771 } } peers { permanent_uuid: "36cb15aca8b54ba9b7509260d9483543" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36521 } } }
I20260811 05:12:46.654680 32176 tablet_replica.cc:442] T ebefde6c00494c1899b4bf0f590b5d6c P 59c38762b52b47b2a417ceb0cee2f0b1: This TxnStatusTablet replica's current role is: LEADER
I20260811 05:12:46.655181 32190 mvcc.cc:205] Tried to move back new op lower bound from 7317197482601635840 to 7317197482517504000. Current Snapshot: MvccSnapshot[applied={T|T < 7317197482601635840}]
I20260811 05:12:46.655514 32230 tablet_replica.cc:439] T ebefde6c00494c1899b4bf0f590b5d6c P 412b9bad9194413faeeda0d15b3cdb2d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41625 } } peers { permanent_uuid: "412b9bad9194413faeeda0d15b3cdb2d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44771 } } peers { permanent_uuid: "36cb15aca8b54ba9b7509260d9483543" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36521 } } }
I20260811 05:12:46.655619 32230 tablet_replica.cc:442] T ebefde6c00494c1899b4bf0f590b5d6c P 412b9bad9194413faeeda0d15b3cdb2d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:12:46.659632 32157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:46.656455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49604 (local address 127.28.143.131:44771)
0811 05:12:46.656502 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:46.656504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:46.656514 (+ 10us) server_negotiation.cc:407] Connection header received
0811 05:12:46.656589 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:46.656590 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:46.656607 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:46.656709 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:12:46.656856 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:46.657445 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:46.658698 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:46.658959 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:46.659027 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:46.659260 (+ 233us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:12:46.659402 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:12:46.659478 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:46.659541 (+ 63us) server_negotiation.cc:299] Negotiation successful
0811 05:12:46.659565 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 05:12:46.666285 32155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:46.662962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:43875 (local address 127.28.143.130:41625)
0811 05:12:46.663023 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:46.663025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:46.663079 (+ 54us) server_negotiation.cc:407] Connection header received
0811 05:12:46.663149 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:46.663150 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:46.663162 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:46.663211 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:12:46.663445 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:46.663958 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:46.665328 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:46.665534 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:46.665599 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:46.665919 (+ 320us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:12:46.665959 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:12:46.666015 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:46.666192 (+ 177us) server_negotiation.cc:299] Negotiation successful
0811 05:12:46.666218 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 05:12:46.666487 32259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:46.662811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:41625 (local address 127.28.143.131:43875)
0811 05:12:46.663009 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:46.663023 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:46.663094 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:46.663226 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:46.663228 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:46.663229 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 05:12:46.663305 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:46.663307 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:46.664077 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:46.664078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:46.665213 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:46.665218 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:46.665657 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:46.665659 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:46.665743 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:46.665769 (+ 26us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 05:12:46.666033 (+ 264us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 05:12:46.666035 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 05:12:46.666082 (+ 47us) client_negotiation.cc:240] Negotiation successful
0811 05:12:46.666147 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":103,"threads_started":1}
I20260811 05:12:46.669528 31535 raft_consensus.cc:1261] T 694e820b5da34a3d926e9902099aed92 P 36cb15aca8b54ba9b7509260d9483543 [term 1 FOLLOWER]: Refusing update from remote peer 412b9bad9194413faeeda0d15b3cdb2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:12:46.669646 31791 raft_consensus.cc:1261] T 694e820b5da34a3d926e9902099aed92 P 59c38762b52b47b2a417ceb0cee2f0b1 [term 1 FOLLOWER]: Refusing update from remote peer 412b9bad9194413faeeda0d15b3cdb2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:12:46.669790 32230 consensus_queue.cc:1035] T 694e820b5da34a3d926e9902099aed92 P 412b9bad9194413faeeda0d15b3cdb2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "36cb15aca8b54ba9b7509260d9483543" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:12:46.669945 32177 consensus_queue.cc:1035] T 694e820b5da34a3d926e9902099aed92 P 412b9bad9194413faeeda0d15b3cdb2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "59c38762b52b47b2a417ceb0cee2f0b1" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:12:46.679795 32157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:46.676653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:40837 (local address 127.28.143.131:44771)
0811 05:12:46.676694 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:46.676696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:46.676951 (+ 255us) server_negotiation.cc:407] Connection header received
0811 05:12:46.677004 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:46.677005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:46.677016 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:46.677084 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:12:46.677223 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:46.677691 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:46.678905 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:46.679230 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:46.679271 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:46.679527 (+ 256us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:12:46.679569 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:12:46.679627 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:46.679698 (+ 71us) server_negotiation.cc:299] Negotiation successful
0811 05:12:46.679722 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 05:12:46.679842 32267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:46.676589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.131:44771 (local address 127.28.143.130:40837)
0811 05:12:46.676876 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:46.676887 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:46.676952 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:46.677096 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:46.677096 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:46.677098 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 05:12:46.677154 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:46.677155 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:46.677758 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:46.677758 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:46.678802 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:46.678805 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:46.679327 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:46.679328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:46.679389 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:46.679410 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 05:12:46.679643 (+ 233us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 05:12:46.679644 (+ 1us) client_negotiation.cc:769] Sending connection context
0811 05:12:46.679694 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 05:12:46.679746 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":117,"threads_started":1}
I20260811 05:12:46.690404 31224 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54838:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:12:46.690548 31224 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54838:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:12:46.691334 31224 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8798dcd480634809ac8bfa3f15e20049: Sending DeleteTablet for 3 replicas of tablet 694e820b5da34a3d926e9902099aed92
I20260811 05:12:46.691674 31501 tablet_service.cc:1514] Processing DeleteTablet for tablet 694e820b5da34a3d926e9902099aed92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:12:46 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:56450
I20260811 05:12:46.691718 32002 tablet_service.cc:1514] Processing DeleteTablet for tablet 694e820b5da34a3d926e9902099aed92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:12:46 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:49588
I20260811 05:12:46.691779 32216 tablet_replica.cc:330] T 694e820b5da34a3d926e9902099aed92 P 36cb15aca8b54ba9b7509260d9483543: stopping tablet replica
I20260811 05:12:46.691815 32214 tablet_replica.cc:330] T 694e820b5da34a3d926e9902099aed92 P 412b9bad9194413faeeda0d15b3cdb2d: stopping tablet replica
I20260811 05:12:46.691828 32216 raft_consensus.cc:2229] T 694e820b5da34a3d926e9902099aed92 P 36cb15aca8b54ba9b7509260d9483543 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:12:46.691857 32214 raft_consensus.cc:2229] T 694e820b5da34a3d926e9902099aed92 P 412b9bad9194413faeeda0d15b3cdb2d [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:12:46.691867 32216 raft_consensus.cc:2258] T 694e820b5da34a3d926e9902099aed92 P 36cb15aca8b54ba9b7509260d9483543 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:12:46.691952 32214 raft_consensus.cc:2258] T 694e820b5da34a3d926e9902099aed92 P 412b9bad9194413faeeda0d15b3cdb2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:12:46.692194 31761 tablet_service.cc:1514] Processing DeleteTablet for tablet 694e820b5da34a3d926e9902099aed92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:12:46 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:41018
I20260811 05:12:46.692293 32207 tablet_replica.cc:330] T 694e820b5da34a3d926e9902099aed92 P 59c38762b52b47b2a417ceb0cee2f0b1: stopping tablet replica
I20260811 05:12:46.692341 32207 raft_consensus.cc:2229] T 694e820b5da34a3d926e9902099aed92 P 59c38762b52b47b2a417ceb0cee2f0b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:12:46.692376 32207 raft_consensus.cc:2258] T 694e820b5da34a3d926e9902099aed92 P 59c38762b52b47b2a417ceb0cee2f0b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:12:46.692674 32216 ts_tablet_manager.cc:1905] T 694e820b5da34a3d926e9902099aed92 P 36cb15aca8b54ba9b7509260d9483543: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:12:46.692705 32207 ts_tablet_manager.cc:1905] T 694e820b5da34a3d926e9902099aed92 P 59c38762b52b47b2a417ceb0cee2f0b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:12:46.694505 32214 ts_tablet_manager.cc:1905] T 694e820b5da34a3d926e9902099aed92 P 412b9bad9194413faeeda0d15b3cdb2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:12:46.694787 32216 ts_tablet_manager.cc:1918] T 694e820b5da34a3d926e9902099aed92 P 36cb15aca8b54ba9b7509260d9483543: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 05:12:46.694844 32216 log.cc:1196] T 694e820b5da34a3d926e9902099aed92 P 36cb15aca8b54ba9b7509260d9483543: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-0/wal/wals/694e820b5da34a3d926e9902099aed92
I20260811 05:12:46.695041 32216 ts_tablet_manager.cc:1939] T 694e820b5da34a3d926e9902099aed92 P 36cb15aca8b54ba9b7509260d9483543: Deleting consensus metadata
I20260811 05:12:46.696575 32214 ts_tablet_manager.cc:1918] T 694e820b5da34a3d926e9902099aed92 P 412b9bad9194413faeeda0d15b3cdb2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 05:12:46.696667 32214 log.cc:1196] T 694e820b5da34a3d926e9902099aed92 P 412b9bad9194413faeeda0d15b3cdb2d: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-2/wal/wals/694e820b5da34a3d926e9902099aed92
I20260811 05:12:46.696913 32214 ts_tablet_manager.cc:1939] T 694e820b5da34a3d926e9902099aed92 P 412b9bad9194413faeeda0d15b3cdb2d: Deleting consensus metadata
I20260811 05:12:46.697216 31199 catalog_manager.cc:4841] TS 36cb15aca8b54ba9b7509260d9483543 (127.28.143.129:36521): tablet 694e820b5da34a3d926e9902099aed92 (table test-table [id=d699fe7d87e6413ebe85f1a2dce1a9f3]) successfully deleted
I20260811 05:12:46.697798 32207 ts_tablet_manager.cc:1918] T 694e820b5da34a3d926e9902099aed92 P 59c38762b52b47b2a417ceb0cee2f0b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 05:12:46.697885 32207 log.cc:1196] T 694e820b5da34a3d926e9902099aed92 P 59c38762b52b47b2a417ceb0cee2f0b1: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425159060450-29246-0/minicluster-data/ts-1/wal/wals/694e820b5da34a3d926e9902099aed92
I20260811 05:12:46.698076 32207 ts_tablet_manager.cc:1939] T 694e820b5da34a3d926e9902099aed92 P 59c38762b52b47b2a417ceb0cee2f0b1: Deleting consensus metadata
I20260811 05:12:46.698415 31204 catalog_manager.cc:4841] TS 59c38762b52b47b2a417ceb0cee2f0b1 (127.28.143.130:41625): tablet 694e820b5da34a3d926e9902099aed92 (table test-table [id=d699fe7d87e6413ebe85f1a2dce1a9f3]) successfully deleted
I20260811 05:12:46.698907 31199 catalog_manager.cc:4841] TS 412b9bad9194413faeeda0d15b3cdb2d (127.28.143.131:44771): tablet 694e820b5da34a3d926e9902099aed92 (table test-table [id=d699fe7d87e6413ebe85f1a2dce1a9f3]) successfully deleted
Aug 11 05:12:46 dist-test-slave-f7th krb5kdc[31141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425165, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.28.143.129@KRBTEST.COM
I20260811 05:12:46.704536 32153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:46.693123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56458 (local address 127.28.143.129:36521)
0811 05:12:46.693168 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:46.693170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:46.693453 (+ 283us) server_negotiation.cc:407] Connection header received
0811 05:12:46.693550 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:46.693551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:46.693567 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:46.693596 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:46.694394 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:46.695124 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:46.696290 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:46.696637 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:46.696692 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:46.700953 (+ 4261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:46.700969 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:46.700969 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:46.700981 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:46.703058 (+ 2077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:46.703506 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:46.703510 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:46.703510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:46.703582 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:46.703784 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:46.703785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:46.703785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:46.704019 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:46.704268 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:46.704325 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 05:12:46.704424 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260811 05:12:46.704870 31609 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56458
I20260811 05:12:46.714115 31351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:46.706198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54860 (local address 127.28.143.190:34053)
0811 05:12:46.706247 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:46.706249 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:46.706261 (+ 12us) server_negotiation.cc:407] Connection header received
0811 05:12:46.706327 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:46.706328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:46.706339 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:46.706409 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:46.707054 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:46.707762 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:46.708859 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:46.709177 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:46.709245 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:46.710908 (+ 1663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:46.710910 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:46.710911 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:46.710922 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:46.712770 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:46.713230 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:46.713231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:46.713231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:46.713306 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:46.713528 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:46.713529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:46.713529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:46.713687 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:46.713796 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:46.713925 (+ 129us) server_negotiation.cc:299] Negotiation successful
0811 05:12:46.714017 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260811 05:12:46.714490 31224 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54860
I20260811 05:12:46.715365 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 31363
I20260811 05:12:46.761695 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 31633
I20260811 05:12:46.788478 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 31880
I20260811 05:12:46.799924 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 31167
2026-08-11T05:12:46Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2930 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:12:46 dist-test-slave-f7th krb5kdc[32291](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:12:46 dist-test-slave-f7th krb5kdc[32291](info): set up 2 sockets
Aug 11 05:12:46 dist-test-slave-f7th krb5kdc[32291](info): commencing operation
krb5kdc: starting...
W20260811 05:12:48.036933 29246 mini_kdc.cc:121] Time spent starting KDC: real 1.201s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Aug 11 05:12:48 dist-test-slave-f7th krb5kdc[32291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:12:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:12:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
I20260811 05:12:48.171564 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:44957
--webserver_interface=127.28.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:34715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:44957
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:48.258131 32313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:48.258366 32313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:48.258399 32313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:48.259603 32313 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:12:48.259651 32313 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 05:12:48.259667 32313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:48.259686 32313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:48.259701 32313 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:12:48.259717 32313 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:12:48.261305 32313 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:34715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:44957
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:44957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:12:48.261564 32313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:48.261783 32313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:48.266791 32324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:48.266813 32326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:48.267023 32313 server_base.cc:1034] running on GCE node
W20260811 05:12:48.266809 32332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:48.267365 32313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:48.267669 32313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:48.268819 32313 hybrid_clock.cc:648] HybridClock initialized: now 1786425168268801 us; error 27 us; skew 500 ppm
Aug 11 05:12:48 dist-test-slave-f7th krb5kdc[32291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425168, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:48.271355 32313 init.cc:377] Logged in from keytab as kudu/127.28.143.190@KRBTEST.COM (short username kudu)
I20260811 05:12:48.271849 32313 webserver.cc:460] Webserver started at http://127.28.143.190:40203/ using document root <none> and password file <none>
I20260811 05:12:48.272081 32313 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:48.272133 32313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:48.272259 32313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:48.273373 32313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "251ba2f17a774486bcc0fa8cfea5c90f"
format_stamp: "Formatted at 2026-08-11 05:12:48 on dist-test-slave-f7th"
server_key: "19d2e9f81a7980288ab6a7657a36b5a7"
server_key_iv: "68b715ddaa608c7a75a60020fc797cb6"
server_key_version: "encryptionkey@0"
I20260811 05:12:48.273716 32313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "251ba2f17a774486bcc0fa8cfea5c90f"
format_stamp: "Formatted at 2026-08-11 05:12:48 on dist-test-slave-f7th"
server_key: "19d2e9f81a7980288ab6a7657a36b5a7"
server_key_iv: "68b715ddaa608c7a75a60020fc797cb6"
server_key_version: "encryptionkey@0"
I20260811 05:12:48.275167 32313 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 05:12:48.276080 32342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:48.276299 32313 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:12:48.276417 32313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "251ba2f17a774486bcc0fa8cfea5c90f"
format_stamp: "Formatted at 2026-08-11 05:12:48 on dist-test-slave-f7th"
server_key: "19d2e9f81a7980288ab6a7657a36b5a7"
server_key_iv: "68b715ddaa608c7a75a60020fc797cb6"
server_key_version: "encryptionkey@0"
I20260811 05:12:48.276499 32313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:48.304949 32313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:48.306974 32313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:48.307120 32313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:48.311980 32313 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:44957
I20260811 05:12:48.312039 32457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:44957 every 8 connection(s)
I20260811 05:12:48.312382 32313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:12:48.313143 32459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:48.315661 32459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 251ba2f17a774486bcc0fa8cfea5c90f: Bootstrap starting.
I20260811 05:12:48.316344 32459 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 251ba2f17a774486bcc0fa8cfea5c90f: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:48.316682 32459 log.cc:824] T 00000000000000000000000000000000 P 251ba2f17a774486bcc0fa8cfea5c90f: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:48.317559 32459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 251ba2f17a774486bcc0fa8cfea5c90f: No bootstrap required, opened a new log
I20260811 05:12:48.318130 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 32313
I20260811 05:12:48.318212 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:12:48.318357 29246 external_mini_cluster.cc:1428] Setting key 33f8c3d23053aa02a09c8d4f501c9f8d
I20260811 05:12:48.320360 32459 raft_consensus.cc:348] T 00000000000000000000000000000000 P 251ba2f17a774486bcc0fa8cfea5c90f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "251ba2f17a774486bcc0fa8cfea5c90f" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 44957 } }
I20260811 05:12:48.320565 32459 raft_consensus.cc:374] T 00000000000000000000000000000000 P 251ba2f17a774486bcc0fa8cfea5c90f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:48.320605 32459 raft_consensus.cc:729] T 00000000000000000000000000000000 P 251ba2f17a774486bcc0fa8cfea5c90f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 251ba2f17a774486bcc0fa8cfea5c90f, State: Initialized, Role: FOLLOWER
I20260811 05:12:48.320778 32459 consensus_queue.cc:260] T 00000000000000000000000000000000 P 251ba2f17a774486bcc0fa8cfea5c90f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "251ba2f17a774486bcc0fa8cfea5c90f" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 44957 } }
I20260811 05:12:48.320854 32459 raft_consensus.cc:388] T 00000000000000000000000000000000 P 251ba2f17a774486bcc0fa8cfea5c90f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:12:48.320886 32459 raft_consensus.cc:482] T 00000000000000000000000000000000 P 251ba2f17a774486bcc0fa8cfea5c90f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:12:48.320937 32459 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 251ba2f17a774486bcc0fa8cfea5c90f [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:48.321693 32459 raft_consensus.cc:504] T 00000000000000000000000000000000 P 251ba2f17a774486bcc0fa8cfea5c90f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "251ba2f17a774486bcc0fa8cfea5c90f" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 44957 } }
I20260811 05:12:48.321815 32459 leader_election.cc:302] T 00000000000000000000000000000000 P 251ba2f17a774486bcc0fa8cfea5c90f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 251ba2f17a774486bcc0fa8cfea5c90f; no voters:
I20260811 05:12:48.322028 32459 leader_election.cc:288] T 00000000000000000000000000000000 P 251ba2f17a774486bcc0fa8cfea5c90f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:12:48.322178 32470 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 251ba2f17a774486bcc0fa8cfea5c90f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:48.322299 32459 sys_catalog.cc:564] T 00000000000000000000000000000000 P 251ba2f17a774486bcc0fa8cfea5c90f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:12:48.322346 32470 raft_consensus.cc:686] T 00000000000000000000000000000000 P 251ba2f17a774486bcc0fa8cfea5c90f [term 1 LEADER]: Becoming Leader. State: Replica: 251ba2f17a774486bcc0fa8cfea5c90f, State: Running, Role: LEADER
I20260811 05:12:48.322445 32470 consensus_queue.cc:237] T 00000000000000000000000000000000 P 251ba2f17a774486bcc0fa8cfea5c90f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "251ba2f17a774486bcc0fa8cfea5c90f" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 44957 } }
I20260811 05:12:48.322868 32471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 251ba2f17a774486bcc0fa8cfea5c90f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "251ba2f17a774486bcc0fa8cfea5c90f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "251ba2f17a774486bcc0fa8cfea5c90f" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 44957 } } }
I20260811 05:12:48.322963 32471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 251ba2f17a774486bcc0fa8cfea5c90f [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:48.323206 32472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 251ba2f17a774486bcc0fa8cfea5c90f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 251ba2f17a774486bcc0fa8cfea5c90f. Latest consensus state: current_term: 1 leader_uuid: "251ba2f17a774486bcc0fa8cfea5c90f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "251ba2f17a774486bcc0fa8cfea5c90f" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 44957 } } }
I20260811 05:12:48.323278 32472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 251ba2f17a774486bcc0fa8cfea5c90f [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:48.323662 32481 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:12:48.324301 32481 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 11 05:12:48 dist-test-slave-f7th krb5kdc[32291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:12:48.329172 32481 catalog_manager.cc:1347] Generated new cluster ID: 5f683cd47598415f9486de7145d3b5d8
I20260811 05:12:48.329248 32481 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:12:48.330905 32468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:48.318894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54198 (local address 127.28.143.190:44957)
0811 05:12:48.319092 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:48.319102 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:48.319166 (+ 64us) server_negotiation.cc:407] Connection header received
0811 05:12:48.320316 (+ 1150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:48.320328 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:48.320412 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:48.320579 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:48.321403 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:48.322789 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:48.323914 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:48.324494 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:48.324609 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:48.327336 (+ 2727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:48.327341 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:48.327341 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:48.327354 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:48.329208 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:48.329603 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:48.329607 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:48.329607 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:48.329734 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:48.329922 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:48.329923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:48.329924 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:48.330199 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:48.330357 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:48.330477 (+ 120us) server_negotiation.cc:299] Negotiation successful
0811 05:12:48.330733 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":80,"threads_started":1}
I20260811 05:12:48.342695 32481 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:12:48.343281 32481 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:12:48.349392 32481 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 251ba2f17a774486bcc0fa8cfea5c90f: Generated new TSK 0
I20260811 05:12:48.349620 32481 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.143.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab.
Entry for principal kudu/127.28.143.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab.
I20260811 05:12:48.385986 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.129:0
--local_ip_for_outbound_sockets=127.28.143.129
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:44957
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:34715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:48.476243 32501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:48.476490 32501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:48.476518 32501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:48.477774 32501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:48.477833 32501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:48.477864 32501 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 05:12:48.477875 32501 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 05:12:48.477891 32501 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.129
I20260811 05:12:48.479470 32501 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:34715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.143.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.143.190:44957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32501
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.129
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:12:48.479707 32501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:48.479925 32501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:48.485836 32516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:48.485810 32519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:48.485810 32517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:48.485850 32501 server_base.cc:1034] running on GCE node
I20260811 05:12:48.486340 32501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:48.486608 32501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:48.487769 32501 hybrid_clock.cc:648] HybridClock initialized: now 1786425168487741 us; error 45 us; skew 500 ppm
Aug 11 05:12:48 dist-test-slave-f7th krb5kdc[32291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425168, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:48.490805 32501 init.cc:377] Logged in from keytab as kudu/127.28.143.129@KRBTEST.COM (short username kudu)
I20260811 05:12:48.491371 32501 webserver.cc:460] Webserver started at http://127.28.143.129:42239/ using document root <none> and password file <none>
I20260811 05:12:48.491600 32501 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:48.491652 32501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:48.491793 32501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:48.493038 32501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-0/data/instance:
uuid: "6d7acbb7aab64a218850d78341c533f9"
format_stamp: "Formatted at 2026-08-11 05:12:48 on dist-test-slave-f7th"
server_key: "14d358bb10f2899ece4356bd374297ed"
server_key_iv: "663d0e5e62946d472c84b8ef4f460d0b"
server_key_version: "encryptionkey@0"
I20260811 05:12:48.493412 32501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance:
uuid: "6d7acbb7aab64a218850d78341c533f9"
format_stamp: "Formatted at 2026-08-11 05:12:48 on dist-test-slave-f7th"
server_key: "14d358bb10f2899ece4356bd374297ed"
server_key_iv: "663d0e5e62946d472c84b8ef4f460d0b"
server_key_version: "encryptionkey@0"
I20260811 05:12:48.494915 32501 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 05:12:48.495950 32529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:48.496239 32501 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:12:48.496443 32501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-0/wal
uuid: "6d7acbb7aab64a218850d78341c533f9"
format_stamp: "Formatted at 2026-08-11 05:12:48 on dist-test-slave-f7th"
server_key: "14d358bb10f2899ece4356bd374297ed"
server_key_iv: "663d0e5e62946d472c84b8ef4f460d0b"
server_key_version: "encryptionkey@0"
I20260811 05:12:48.496536 32501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:48.513751 32501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:48.516248 32501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:48.516410 32501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:48.517091 32501 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:12:48.517140 32501 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:48.517175 32501 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:12:48.517213 32501 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:48.524760 32501 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.129:43067
Aug 11 05:12:48 dist-test-slave-f7th krb5kdc[32291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425168, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:12:48.524830 32742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.129:43067 every 8 connection(s)
I20260811 05:12:48.525194 32501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
I20260811 05:12:48.528142 32468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:48.517525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:41499 (local address 127.28.143.190:44957)
0811 05:12:48.517596 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:48.517598 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:48.517831 (+ 233us) server_negotiation.cc:407] Connection header received
0811 05:12:48.518562 (+ 731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:48.518564 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:48.518586 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:48.518666 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:48.519574 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:48.520118 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:48.521777 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:48.522122 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:48.522237 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:48.525338 (+ 3101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:48.525340 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:48.525341 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:48.525350 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:48.526733 (+ 1383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:48.527172 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:48.527173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:48.527174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:48.527236 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:48.527472 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:48.527472 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:48.527473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:48.527605 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:48.527663 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:48.527949 (+ 286us) server_negotiation.cc:299] Negotiation successful
0811 05:12:48.528050 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 05:12:48.528299 32545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:48.517502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:44957 (local address 127.28.143.129:41499)
0811 05:12:48.517678 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:48.517733 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:48.518422 (+ 689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:48.518752 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:48.518757 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:48.519144 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:48.519475 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:48.519481 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:48.520205 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:48.520206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:48.521604 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:48.521609 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:48.522265 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:48.522267 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:48.522425 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:48.523510 (+ 1085us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:12:48.523521 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:48.525226 (+ 1705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:48.526845 (+ 1619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:48.526854 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:48.526856 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:48.527094 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:48.527292 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:48.527293 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:48.527293 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:48.527413 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:48.527662 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:48.527669 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:48.527812 (+ 143us) client_negotiation.cc:769] Sending connection context
0811 05:12:48.527904 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 05:12:48.528077 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":91,"threads_started":1}
I20260811 05:12:48.528831 32743 heartbeater.cc:347] Connected to a master server at 127.28.143.190:44957
I20260811 05:12:48.528923 32743 heartbeater.cc:464] Registering TS with master...
I20260811 05:12:48.529109 32743 heartbeater.cc:511] Master 127.28.143.190:44957 requested a full tablet report, sending...
I20260811 05:12:48.529709 32375 ts_manager.cc:194] Registered new tserver with Master: 6d7acbb7aab64a218850d78341c533f9 (127.28.143.129:43067)
I20260811 05:12:48.530576 32375 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.129@KRBTEST.COM'} at 127.28.143.129:41499
I20260811 05:12:48.532838 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 32501
I20260811 05:12:48.532912 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance
I20260811 05:12:48.533090 29246 external_mini_cluster.cc:1428] Setting key 3ef972913ad8a3b4e4697c971d68bdc7
I20260811 05:12:48.535949 32468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:48.530088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:44501 (local address 127.28.143.190:44957)
0811 05:12:48.530130 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:48.530132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:48.530197 (+ 65us) server_negotiation.cc:407] Connection header received
0811 05:12:48.530261 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:48.530261 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:48.530271 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:48.530319 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:48.530648 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:48.531162 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:48.531905 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:48.532108 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:48.532172 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:48.533241 (+ 1069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:48.533243 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:48.533243 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:48.533251 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:48.534823 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:48.535242 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:48.535243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:48.535243 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:48.535308 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:48.535466 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:48.535467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:48.535467 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:48.535589 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:48.535643 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:48.535782 (+ 139us) server_negotiation.cc:299] Negotiation successful
0811 05:12:48.535864 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 05:12:48.535957 32753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:48.530007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:44957 (local address 127.28.143.129:44501)
0811 05:12:48.530142 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:48.530153 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:48.530205 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:48.530327 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:48.530327 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:48.530519 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:48.530578 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:48.530580 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:48.531246 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:48.531247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:48.531812 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:48.531814 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:48.532199 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:48.532200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:48.532275 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:48.532769 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:12:48.532774 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:48.533165 (+ 391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:48.534942 (+ 1777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:48.534943 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:48.534943 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:48.535166 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:48.535367 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:48.535368 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:48.535368 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:48.535424 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:48.535641 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:48.535642 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:48.535700 (+ 58us) client_negotiation.cc:769] Sending connection context
0811 05:12:48.535752 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 05:12:48.535853 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":66,"threads_started":1}
WARNING: no policy specified for kudu/127.28.143.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab.
Entry for principal kudu/127.28.143.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab.
I20260811 05:12:48.561102 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.130:0
--local_ip_for_outbound_sockets=127.28.143.130
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:44957
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:34715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:48.647954 32757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:48.648176 32757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:48.648202 32757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:48.649507 32757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:48.649564 32757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:48.649585 32757 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 05:12:48.649596 32757 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 05:12:48.649613 32757 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.130
I20260811 05:12:48.651212 32757 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:34715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.143.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.143.190:44957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.130
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:12:48.651463 32757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:48.651700 32757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:48.657335 300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:48.657328 304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:48.657560 32757 server_base.cc:1034] running on GCE node
W20260811 05:12:48.657454 32764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:48.657857 32757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:48.658131 32757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:48.659304 32757 hybrid_clock.cc:648] HybridClock initialized: now 1786425168659281 us; error 39 us; skew 500 ppm
Aug 11 05:12:48 dist-test-slave-f7th krb5kdc[32291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425168, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:48.662211 32757 init.cc:377] Logged in from keytab as kudu/127.28.143.130@KRBTEST.COM (short username kudu)
I20260811 05:12:48.662732 32757 webserver.cc:460] Webserver started at http://127.28.143.130:37217/ using document root <none> and password file <none>
I20260811 05:12:48.662962 32757 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:48.663004 32757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:48.663107 32757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:48.664198 32757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-1/data/instance:
uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5"
format_stamp: "Formatted at 2026-08-11 05:12:48 on dist-test-slave-f7th"
server_key: "e43617f42f21803c1c2825e3bdc60178"
server_key_iv: "7dbdc1a65905457baa2ef664051583f7"
server_key_version: "encryptionkey@0"
I20260811 05:12:48.664529 32757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance:
uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5"
format_stamp: "Formatted at 2026-08-11 05:12:48 on dist-test-slave-f7th"
server_key: "e43617f42f21803c1c2825e3bdc60178"
server_key_iv: "7dbdc1a65905457baa2ef664051583f7"
server_key_version: "encryptionkey@0"
I20260811 05:12:48.665910 32757 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 05:12:48.666708 319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:48.666991 32757 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 05:12:48.667080 32757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-1/wal
uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5"
format_stamp: "Formatted at 2026-08-11 05:12:48 on dist-test-slave-f7th"
server_key: "e43617f42f21803c1c2825e3bdc60178"
server_key_iv: "7dbdc1a65905457baa2ef664051583f7"
server_key_version: "encryptionkey@0"
I20260811 05:12:48.667148 32757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:48.681625 32757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:48.684046 32757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:48.684180 32757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:48.684777 32757 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:12:48.684826 32757 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:48.684849 32757 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:12:48.684861 32757 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:48.691200 32757 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.130:37639
I20260811 05:12:48.691351 519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.130:37639 every 8 connection(s)
I20260811 05:12:48.691627 32757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
Aug 11 05:12:48 dist-test-slave-f7th krb5kdc[32291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425168, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:12:48.695374 32468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:48.685232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:50269 (local address 127.28.143.190:44957)
0811 05:12:48.685301 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:48.685303 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:48.685583 (+ 280us) server_negotiation.cc:407] Connection header received
0811 05:12:48.686150 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:48.686151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:48.686164 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:48.686246 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:48.687183 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:48.687978 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:48.689575 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:48.689881 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:48.689963 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:48.692787 (+ 2824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:48.692789 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:48.692789 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:48.692799 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:48.693982 (+ 1183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:48.694407 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:48.694409 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:48.694410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:48.694470 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:48.694703 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:48.694704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:48.694704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:48.694835 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:48.694894 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:48.695186 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 05:12:48.695281 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 05:12:48.695554 342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:48.685190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:44957 (local address 127.28.143.130:50269)
0811 05:12:48.685473 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:48.685505 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:48.686018 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:48.686325 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:48.686330 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:48.686754 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:48.687066 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:48.687073 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:48.688091 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:48.688092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:48.689441 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:48.689446 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:48.690022 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:48.690023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:48.690196 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:48.690855 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:12:48.690865 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:48.692682 (+ 1817us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:48.694085 (+ 1403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:48.694094 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:48.694097 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:48.694331 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:48.694532 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:48.694533 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:48.694534 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:48.694646 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:48.694894 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:48.694902 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:48.695056 (+ 154us) client_negotiation.cc:769] Sending connection context
0811 05:12:48.695152 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 05:12:48.695324 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":82,"threads_started":1}
I20260811 05:12:48.696095 522 heartbeater.cc:347] Connected to a master server at 127.28.143.190:44957
I20260811 05:12:48.696187 522 heartbeater.cc:464] Registering TS with master...
I20260811 05:12:48.696370 522 heartbeater.cc:511] Master 127.28.143.190:44957 requested a full tablet report, sending...
I20260811 05:12:48.696861 32377 ts_manager.cc:194] Registered new tserver with Master: c80c0a0fe54e4382abe4e7a0345fe9f5 (127.28.143.130:37639)
I20260811 05:12:48.697341 32377 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.130@KRBTEST.COM'} at 127.28.143.130:50269
I20260811 05:12:48.697886 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 32757
I20260811 05:12:48.697983 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance
I20260811 05:12:48.698179 29246 external_mini_cluster.cc:1428] Setting key ce1c3dde050baa1636020fc997ec2b52
I20260811 05:12:48.703313 537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:48.697160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:44957 (local address 127.28.143.130:44695)
0811 05:12:48.697303 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:48.697314 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:48.697357 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:48.697588 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:48.697589 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:48.697793 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:48.697856 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:48.697857 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:48.698638 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:48.698639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:48.699147 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:48.699149 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:48.699485 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:48.699486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:48.699580 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:48.700040 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:12:48.700045 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:48.700404 (+ 359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:48.702048 (+ 1644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:48.702048 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:48.702049 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:48.702227 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:48.702457 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:48.702458 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:48.702458 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:48.702527 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:48.702907 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:48.702908 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:48.703042 (+ 134us) client_negotiation.cc:769] Sending connection context
0811 05:12:48.703122 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 05:12:48.703230 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":73,"threads_started":1}
I20260811 05:12:48.703366 32468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:48.697230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:44695 (local address 127.28.143.190:44957)
0811 05:12:48.697455 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:48.697457 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:48.697466 (+ 9us) server_negotiation.cc:407] Connection header received
0811 05:12:48.697485 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:48.697485 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:48.697494 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:48.697561 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:48.697932 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:48.698553 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:48.699219 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:48.699426 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:48.699481 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:48.700542 (+ 1061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:48.700544 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:48.700544 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:48.700551 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:48.701954 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:48.702300 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:48.702301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:48.702302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:48.702370 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:48.702592 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:48.702593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:48.702593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:48.702807 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:48.702877 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:48.703176 (+ 299us) server_negotiation.cc:299] Negotiation successful
0811 05:12:48.703274 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205}
WARNING: no policy specified for kudu/127.28.143.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab.
Entry for principal kudu/127.28.143.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab.
I20260811 05:12:48.727289 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.131:0
--local_ip_for_outbound_sockets=127.28.143.131
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:44957
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:34715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:48.816017 541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:48.816264 541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:48.816293 541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:48.817570 541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:48.817618 541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:48.817638 541 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 05:12:48.817649 541 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 05:12:48.817665 541 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.131
I20260811 05:12:48.819173 541 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:34715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.143.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.143.190:44957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.541
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.131
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:12:48.819401 541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:48.819608 541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:48.825095 557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:48.825103 551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:48.825111 549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:48.825465 541 server_base.cc:1034] running on GCE node
I20260811 05:12:48.825816 541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:48.826089 541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:48.827275 541 hybrid_clock.cc:648] HybridClock initialized: now 1786425168827248 us; error 42 us; skew 500 ppm
Aug 11 05:12:48 dist-test-slave-f7th krb5kdc[32291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425168, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:48.830367 541 init.cc:377] Logged in from keytab as kudu/127.28.143.131@KRBTEST.COM (short username kudu)
I20260811 05:12:48.830984 541 webserver.cc:460] Webserver started at http://127.28.143.131:38071/ using document root <none> and password file <none>
I20260811 05:12:48.831244 541 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:48.831296 541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:48.831409 541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:48.832558 541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-2/data/instance:
uuid: "d35b5b28f83c408ab44e03ffd27c81e4"
format_stamp: "Formatted at 2026-08-11 05:12:48 on dist-test-slave-f7th"
server_key: "e6d056a4e0704cd225cbe102afc62197"
server_key_iv: "d2f5d09b8144b487001a84201b97138d"
server_key_version: "encryptionkey@0"
I20260811 05:12:48.833015 541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance:
uuid: "d35b5b28f83c408ab44e03ffd27c81e4"
format_stamp: "Formatted at 2026-08-11 05:12:48 on dist-test-slave-f7th"
server_key: "e6d056a4e0704cd225cbe102afc62197"
server_key_iv: "d2f5d09b8144b487001a84201b97138d"
server_key_version: "encryptionkey@0"
I20260811 05:12:48.834542 541 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:12:48.835546 571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:48.835755 541 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:12:48.835879 541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-2/wal
uuid: "d35b5b28f83c408ab44e03ffd27c81e4"
format_stamp: "Formatted at 2026-08-11 05:12:48 on dist-test-slave-f7th"
server_key: "e6d056a4e0704cd225cbe102afc62197"
server_key_iv: "d2f5d09b8144b487001a84201b97138d"
server_key_version: "encryptionkey@0"
I20260811 05:12:48.835953 541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:48.844949 541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:48.847647 541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:48.847811 541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:48.848407 541 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:12:48.848453 541 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:48.848483 541 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:12:48.848500 541 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:48.855461 541 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.131:38799
I20260811 05:12:48.855666 780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.131:38799 every 8 connection(s)
I20260811 05:12:48.855933 541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
Aug 11 05:12:48 dist-test-slave-f7th krb5kdc[32291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425168, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:12:48.859818 32468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:48.848979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:34581 (local address 127.28.143.190:44957)
0811 05:12:48.849039 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:48.849041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:48.849315 (+ 274us) server_negotiation.cc:407] Connection header received
0811 05:12:48.849974 (+ 659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:48.849976 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:48.849992 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:48.850057 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:48.850983 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:48.851773 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:48.853440 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:48.853654 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:48.853715 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:48.856945 (+ 3230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:48.856947 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:48.856948 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:48.856958 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:48.858358 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:48.858817 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:48.858818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:48.858819 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:48.858880 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:48.859134 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:48.859135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:48.859136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:48.859263 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:48.859324 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:48.859621 (+ 297us) server_negotiation.cc:299] Negotiation successful
0811 05:12:48.859722 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 05:12:48.859999 580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:48.848961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:44957 (local address 127.28.143.131:34581)
0811 05:12:48.849167 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:48.849213 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:48.849802 (+ 589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:48.850142 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:48.850147 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:48.850540 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:48.850848 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:48.850854 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:48.851907 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:48.851908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:48.853310 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:48.853315 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:48.853745 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:48.853747 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:48.853886 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:48.854530 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:12:48.854542 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:48.856810 (+ 2268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:48.858462 (+ 1652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:48.858473 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:48.858477 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:48.858736 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:48.858941 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:48.858942 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:48.858942 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:48.859073 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:48.859322 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:48.859331 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:48.859487 (+ 156us) client_negotiation.cc:769] Sending connection context
0811 05:12:48.859589 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 05:12:48.859764 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":120,"threads_started":1}
I20260811 05:12:48.860515 781 heartbeater.cc:347] Connected to a master server at 127.28.143.190:44957
I20260811 05:12:48.860649 781 heartbeater.cc:464] Registering TS with master...
I20260811 05:12:48.860854 781 heartbeater.cc:511] Master 127.28.143.190:44957 requested a full tablet report, sending...
I20260811 05:12:48.861400 32375 ts_manager.cc:194] Registered new tserver with Master: d35b5b28f83c408ab44e03ffd27c81e4 (127.28.143.131:38799)
I20260811 05:12:48.861871 32375 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.131@KRBTEST.COM'} at 127.28.143.131:34581
I20260811 05:12:48.863694 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 541
I20260811 05:12:48.863788 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance
I20260811 05:12:48.863991 29246 external_mini_cluster.cc:1428] Setting key ccfa7c8eca5a66f80fe1cb2885ec0bbd
I20260811 05:12:48.864966 29246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:12:48.869417 32468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:48.862273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:36211 (local address 127.28.143.190:44957)
0811 05:12:48.862323 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:48.862325 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:48.862891 (+ 566us) server_negotiation.cc:407] Connection header received
0811 05:12:48.862998 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:48.863000 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:48.863011 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:48.863081 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:48.863436 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:48.864118 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:48.865147 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:48.865420 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:48.865497 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:48.866572 (+ 1075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:48.866574 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:48.866574 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:48.866582 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:48.868084 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:48.868529 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:48.868531 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:48.868531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:48.868655 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:48.868843 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:48.868844 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:48.868844 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:48.868978 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:48.869048 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:48.869219 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 05:12:48.869311 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 05:12:48.869426 792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:48.862156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:44957 (local address 127.28.143.131:36211)
0811 05:12:48.862797 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:48.862808 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:48.862926 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:48.863086 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:48.863087 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:48.863267 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:48.863324 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:48.863326 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:48.864254 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:48.864255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:48.865074 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:48.865077 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:48.865519 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:48.865521 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:48.865596 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:48.866067 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:12:48.866071 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:48.866460 (+ 389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:48.868173 (+ 1713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:48.868175 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:48.868175 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:48.868419 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:48.868711 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:48.868712 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:48.868712 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:48.868787 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:48.869046 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:48.869047 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:48.869116 (+ 69us) client_negotiation.cc:769] Sending connection context
0811 05:12:48.869184 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 05:12:48.869313 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":80,"threads_started":1}
Aug 11 05:12:48 dist-test-slave-f7th krb5kdc[32291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 05:12:48 dist-test-slave-f7th krb5kdc[32291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:12:48.892305 32468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:48.884592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54204 (local address 127.28.143.190:44957)
0811 05:12:48.884705 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:48.884707 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:48.884715 (+ 8us) server_negotiation.cc:407] Connection header received
0811 05:12:48.884742 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:48.884743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:48.884752 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:48.884800 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:48.885540 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:48.886045 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:48.886793 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:48.886993 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:48.887050 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:48.889651 (+ 2601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:48.889653 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:48.889653 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:48.889662 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:48.891108 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:48.891497 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:48.891498 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:48.891499 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:48.891561 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:48.891736 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:48.891736 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:48.891737 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:48.891864 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:48.891963 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:48.892109 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 05:12:48.892208 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 05:12:48.894008 32375 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54204:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:12:48.895669 32375 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:12:48.902801 813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:48.898342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44154 (local address 127.28.143.130:37639)
0811 05:12:48.898770 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:48.898775 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:48.898786 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:12:48.898830 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:48.898832 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:48.898849 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:48.898957 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:48.899127 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:48.900073 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:48.901539 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:48.902318 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:48.902408 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:48.902482 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:48.902660 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 05:12:48.902703 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":112,"threads_started":1}
I20260811 05:12:48.902781 811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:48.898244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.131:38799 (local address 127.0.0.1:33680)
0811 05:12:48.898383 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:48.898433 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:48.898526 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:48.898990 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:48.898991 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:48.898993 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:48.899087 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:48.899090 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:48.900378 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:48.900379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:48.901422 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:48.901426 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:48.902441 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:48.902443 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:48.902538 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:48.902553 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 05:12:48.902598 (+ 45us) client_negotiation.cc:240] Negotiation successful
0811 05:12:48.902655 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":27,"spinlock_wait_cycles":896,"thread_start_us":91,"threads_started":1}
I20260811 05:12:48.902781 816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:48.898243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:37639 (local address 127.0.0.1:44154)
0811 05:12:48.898587 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:48.898598 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:48.898646 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:48.898954 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:48.898958 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:48.898960 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:48.899061 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:48.899064 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:48.900145 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:48.900146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:48.901429 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:48.901432 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:48.902424 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:48.902426 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:48.902541 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:48.902553 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 05:12:48.902626 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 05:12:48.902675 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":28,"thread_start_us":73,"threads_started":1}
I20260811 05:12:48.903126 817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:48.898603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45924 (local address 127.28.143.129:43067)
0811 05:12:48.898770 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:48.898775 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:48.898870 (+ 95us) server_negotiation.cc:407] Connection header received
0811 05:12:48.899279 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:48.899282 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:48.899297 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:48.899374 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:48.899505 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:48.900603 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:48.901824 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:48.902338 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:48.902462 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:48.902527 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:48.902826 (+ 299us) server_negotiation.cc:299] Negotiation successful
0811 05:12:48.903023 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":83,"threads_started":1}
I20260811 05:12:48.902779 812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:48.898464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:43067 (local address 127.0.0.1:45924)
0811 05:12:48.898796 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:48.898805 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:48.899181 (+ 376us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:48.899394 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:48.899395 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:48.899397 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:48.899436 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:48.899438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:48.900735 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:48.900736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:48.901740 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:48.901743 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:48.902562 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:48.902563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:48.902614 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:48.902617 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 05:12:48.902658 (+ 41us) client_negotiation.cc:240] Negotiation successful
0811 05:12:48.902690 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":75,"threads_started":1}
I20260811 05:12:48.903239 818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:48.898618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33680 (local address 127.28.143.131:38799)
0811 05:12:48.898814 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:48.898818 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:48.898827 (+ 9us) server_negotiation.cc:407] Connection header received
0811 05:12:48.898864 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:48.898865 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:48.898891 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:48.899031 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:48.899177 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:48.900277 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:48.901523 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:48.902318 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:48.903032 (+ 714us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:48.903083 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:48.903105 (+ 22us) server_negotiation.cc:299] Negotiation successful
0811 05:12:48.903142 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":90,"threads_started":1}
I20260811 05:12:48.903838 412 tablet_service.cc:1467] Processing CreateTablet for tablet 5c6ed28a34454f57b41b889ba884d304 (DEFAULT_TABLE table=test-table [id=7fe4524343b04bf8a69a3d13f0e19365]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:12:48.904011 32628 tablet_service.cc:1467] Processing CreateTablet for tablet 5c6ed28a34454f57b41b889ba884d304 (DEFAULT_TABLE table=test-table [id=7fe4524343b04bf8a69a3d13f0e19365]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:12:48.904013 649 tablet_service.cc:1467] Processing CreateTablet for tablet 5c6ed28a34454f57b41b889ba884d304 (DEFAULT_TABLE table=test-table [id=7fe4524343b04bf8a69a3d13f0e19365]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:12:48.904191 412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c6ed28a34454f57b41b889ba884d304. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:48.904374 32628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c6ed28a34454f57b41b889ba884d304. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:48.904374 649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c6ed28a34454f57b41b889ba884d304. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:48.907431 819 tablet_bootstrap.cc:492] T 5c6ed28a34454f57b41b889ba884d304 P c80c0a0fe54e4382abe4e7a0345fe9f5: Bootstrap starting.
I20260811 05:12:48.908059 822 tablet_bootstrap.cc:492] T 5c6ed28a34454f57b41b889ba884d304 P 6d7acbb7aab64a218850d78341c533f9: Bootstrap starting.
I20260811 05:12:48.908147 821 tablet_bootstrap.cc:492] T 5c6ed28a34454f57b41b889ba884d304 P d35b5b28f83c408ab44e03ffd27c81e4: Bootstrap starting.
I20260811 05:12:48.908195 819 tablet_bootstrap.cc:654] T 5c6ed28a34454f57b41b889ba884d304 P c80c0a0fe54e4382abe4e7a0345fe9f5: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:48.908571 819 log.cc:824] T 5c6ed28a34454f57b41b889ba884d304 P c80c0a0fe54e4382abe4e7a0345fe9f5: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:48.908783 822 tablet_bootstrap.cc:654] T 5c6ed28a34454f57b41b889ba884d304 P 6d7acbb7aab64a218850d78341c533f9: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:48.909106 821 tablet_bootstrap.cc:654] T 5c6ed28a34454f57b41b889ba884d304 P d35b5b28f83c408ab44e03ffd27c81e4: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:48.909116 822 log.cc:824] T 5c6ed28a34454f57b41b889ba884d304 P 6d7acbb7aab64a218850d78341c533f9: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:48.909324 819 tablet_bootstrap.cc:492] T 5c6ed28a34454f57b41b889ba884d304 P c80c0a0fe54e4382abe4e7a0345fe9f5: No bootstrap required, opened a new log
I20260811 05:12:48.909409 819 ts_tablet_manager.cc:1397] T 5c6ed28a34454f57b41b889ba884d304 P c80c0a0fe54e4382abe4e7a0345fe9f5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 05:12:48.909427 821 log.cc:824] T 5c6ed28a34454f57b41b889ba884d304 P d35b5b28f83c408ab44e03ffd27c81e4: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:48.909847 822 tablet_bootstrap.cc:492] T 5c6ed28a34454f57b41b889ba884d304 P 6d7acbb7aab64a218850d78341c533f9: No bootstrap required, opened a new log
I20260811 05:12:48.909950 822 ts_tablet_manager.cc:1397] T 5c6ed28a34454f57b41b889ba884d304 P 6d7acbb7aab64a218850d78341c533f9: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260811 05:12:48.910189 821 tablet_bootstrap.cc:492] T 5c6ed28a34454f57b41b889ba884d304 P d35b5b28f83c408ab44e03ffd27c81e4: No bootstrap required, opened a new log
I20260811 05:12:48.910344 821 ts_tablet_manager.cc:1397] T 5c6ed28a34454f57b41b889ba884d304 P d35b5b28f83c408ab44e03ffd27c81e4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260811 05:12:48.911194 819 raft_consensus.cc:348] T 5c6ed28a34454f57b41b889ba884d304 P c80c0a0fe54e4382abe4e7a0345fe9f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } }
I20260811 05:12:48.911342 819 raft_consensus.cc:374] T 5c6ed28a34454f57b41b889ba884d304 P c80c0a0fe54e4382abe4e7a0345fe9f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:48.911384 819 raft_consensus.cc:729] T 5c6ed28a34454f57b41b889ba884d304 P c80c0a0fe54e4382abe4e7a0345fe9f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c80c0a0fe54e4382abe4e7a0345fe9f5, State: Initialized, Role: FOLLOWER
I20260811 05:12:48.911532 819 consensus_queue.cc:260] T 5c6ed28a34454f57b41b889ba884d304 P c80c0a0fe54e4382abe4e7a0345fe9f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } }
I20260811 05:12:48.911808 522 heartbeater.cc:503] Master 127.28.143.190:44957 was elected leader, sending a full tablet report...
I20260811 05:12:48.912078 819 ts_tablet_manager.cc:1428] T 5c6ed28a34454f57b41b889ba884d304 P c80c0a0fe54e4382abe4e7a0345fe9f5: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 05:12:48.912015 822 raft_consensus.cc:348] T 5c6ed28a34454f57b41b889ba884d304 P 6d7acbb7aab64a218850d78341c533f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } }
I20260811 05:12:48.912174 822 raft_consensus.cc:374] T 5c6ed28a34454f57b41b889ba884d304 P 6d7acbb7aab64a218850d78341c533f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:48.912205 822 raft_consensus.cc:729] T 5c6ed28a34454f57b41b889ba884d304 P 6d7acbb7aab64a218850d78341c533f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d7acbb7aab64a218850d78341c533f9, State: Initialized, Role: FOLLOWER
I20260811 05:12:48.912333 822 consensus_queue.cc:260] T 5c6ed28a34454f57b41b889ba884d304 P 6d7acbb7aab64a218850d78341c533f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } }
I20260811 05:12:48.912388 821 raft_consensus.cc:348] T 5c6ed28a34454f57b41b889ba884d304 P d35b5b28f83c408ab44e03ffd27c81e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } }
I20260811 05:12:48.912542 821 raft_consensus.cc:374] T 5c6ed28a34454f57b41b889ba884d304 P d35b5b28f83c408ab44e03ffd27c81e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:48.912590 821 raft_consensus.cc:729] T 5c6ed28a34454f57b41b889ba884d304 P d35b5b28f83c408ab44e03ffd27c81e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35b5b28f83c408ab44e03ffd27c81e4, State: Initialized, Role: FOLLOWER
I20260811 05:12:48.912662 822 ts_tablet_manager.cc:1428] T 5c6ed28a34454f57b41b889ba884d304 P 6d7acbb7aab64a218850d78341c533f9: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260811 05:12:48.912791 32743 heartbeater.cc:503] Master 127.28.143.190:44957 was elected leader, sending a full tablet report...
I20260811 05:12:48.912763 821 consensus_queue.cc:260] T 5c6ed28a34454f57b41b889ba884d304 P d35b5b28f83c408ab44e03ffd27c81e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } }
I20260811 05:12:48.913028 821 ts_tablet_manager.cc:1428] T 5c6ed28a34454f57b41b889ba884d304 P d35b5b28f83c408ab44e03ffd27c81e4: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 05:12:48.913177 781 heartbeater.cc:503] Master 127.28.143.190:44957 was elected leader, sending a full tablet report...
W20260811 05:12:48.942298 526 tablet.cc:2369] T 5c6ed28a34454f57b41b889ba884d304 P c80c0a0fe54e4382abe4e7a0345fe9f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:12:49.021040 836 raft_consensus.cc:482] T 5c6ed28a34454f57b41b889ba884d304 P 6d7acbb7aab64a218850d78341c533f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:12:49.021209 836 raft_consensus.cc:504] T 5c6ed28a34454f57b41b889ba884d304 P 6d7acbb7aab64a218850d78341c533f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } }
I20260811 05:12:49.021625 836 leader_election.cc:288] T 5c6ed28a34454f57b41b889ba884d304 P 6d7acbb7aab64a218850d78341c533f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c80c0a0fe54e4382abe4e7a0345fe9f5 (127.28.143.130:37639), d35b5b28f83c408ab44e03ffd27c81e4 (127.28.143.131:38799)
I20260811 05:12:49.025799 32545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:49.021809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:37639 (local address 127.28.143.129:35205)
0811 05:12:49.021936 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:49.021950 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:49.022002 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:49.022162 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:49.022163 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:49.022164 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:49.022243 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:49.022245 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:49.022980 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:49.022981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:49.024440 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:49.024443 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:49.025504 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:49.025505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:49.025593 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:49.025598 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 05:12:49.025656 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 05:12:49.025708 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":3}
I20260811 05:12:49.025807 813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:49.021989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:35205 (local address 127.28.143.130:37639)
0811 05:12:49.022039 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:49.022041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:49.022051 (+ 10us) server_negotiation.cc:407] Connection header received
0811 05:12:49.022083 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:49.022084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:49.022094 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:49.022151 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:49.022310 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:49.022884 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:49.024740 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:49.025366 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:49.025453 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:49.025465 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:49.025683 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 05:12:49.025736 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 05:12:49.025838 818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:49.021958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:42581 (local address 127.28.143.131:38799)
0811 05:12:49.022036 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:49.022038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:49.022139 (+ 101us) server_negotiation.cc:407] Connection header received
0811 05:12:49.022189 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:49.022190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:49.022198 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:49.022253 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:49.022442 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:49.023033 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:49.024896 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:49.025460 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:49.025562 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:49.025572 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:49.025740 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 05:12:49.025781 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260811 05:12:49.026039 32744 tablet.cc:2369] T 5c6ed28a34454f57b41b889ba884d304 P 6d7acbb7aab64a218850d78341c533f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:12:49.026057 839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:49.021833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.131:38799 (local address 127.28.143.129:42581)
0811 05:12:49.022059 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:49.022070 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:49.022139 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:49.022263 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:49.022264 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:49.022265 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:49.022337 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:49.022339 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:49.023138 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:49.023139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:49.024801 (+ 1662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:49.024804 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:49.025562 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:49.025564 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:49.025649 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:49.025654 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 05:12:49.025915 (+ 261us) client_negotiation.cc:240] Negotiation successful
0811 05:12:49.025962 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":391,"thread_start_us":137,"threads_started":1}
I20260811 05:12:49.026254 449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c6ed28a34454f57b41b889ba884d304" candidate_uuid: "6d7acbb7aab64a218850d78341c533f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" is_pre_election: true
I20260811 05:12:49.026430 449 raft_consensus.cc:2454] T 5c6ed28a34454f57b41b889ba884d304 P c80c0a0fe54e4382abe4e7a0345fe9f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d7acbb7aab64a218850d78341c533f9 in term 0.
I20260811 05:12:49.026717 32536 leader_election.cc:302] T 5c6ed28a34454f57b41b889ba884d304 P 6d7acbb7aab64a218850d78341c533f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d7acbb7aab64a218850d78341c533f9, c80c0a0fe54e4382abe4e7a0345fe9f5; no voters:
I20260811 05:12:49.026870 836 raft_consensus.cc:2781] T 5c6ed28a34454f57b41b889ba884d304 P 6d7acbb7aab64a218850d78341c533f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:12:49.026930 836 raft_consensus.cc:482] T 5c6ed28a34454f57b41b889ba884d304 P 6d7acbb7aab64a218850d78341c533f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:12:49.026955 836 raft_consensus.cc:3037] T 5c6ed28a34454f57b41b889ba884d304 P 6d7acbb7aab64a218850d78341c533f9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:49.027453 689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c6ed28a34454f57b41b889ba884d304" candidate_uuid: "6d7acbb7aab64a218850d78341c533f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" is_pre_election: true
I20260811 05:12:49.027624 689 raft_consensus.cc:2454] T 5c6ed28a34454f57b41b889ba884d304 P d35b5b28f83c408ab44e03ffd27c81e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d7acbb7aab64a218850d78341c533f9 in term 0.
I20260811 05:12:49.027920 836 raft_consensus.cc:504] T 5c6ed28a34454f57b41b889ba884d304 P 6d7acbb7aab64a218850d78341c533f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } }
I20260811 05:12:49.028141 836 leader_election.cc:288] T 5c6ed28a34454f57b41b889ba884d304 P 6d7acbb7aab64a218850d78341c533f9 [CANDIDATE]: Term 1 election: Requested vote from peers c80c0a0fe54e4382abe4e7a0345fe9f5 (127.28.143.130:37639), d35b5b28f83c408ab44e03ffd27c81e4 (127.28.143.131:38799)
I20260811 05:12:49.028314 449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c6ed28a34454f57b41b889ba884d304" candidate_uuid: "6d7acbb7aab64a218850d78341c533f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5"
I20260811 05:12:49.028394 449 raft_consensus.cc:3037] T 5c6ed28a34454f57b41b889ba884d304 P c80c0a0fe54e4382abe4e7a0345fe9f5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:49.029210 449 raft_consensus.cc:2454] T 5c6ed28a34454f57b41b889ba884d304 P c80c0a0fe54e4382abe4e7a0345fe9f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d7acbb7aab64a218850d78341c533f9 in term 1.
I20260811 05:12:49.029433 32536 leader_election.cc:302] T 5c6ed28a34454f57b41b889ba884d304 P 6d7acbb7aab64a218850d78341c533f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d7acbb7aab64a218850d78341c533f9, c80c0a0fe54e4382abe4e7a0345fe9f5; no voters:
I20260811 05:12:49.029518 836 raft_consensus.cc:2781] T 5c6ed28a34454f57b41b889ba884d304 P 6d7acbb7aab64a218850d78341c533f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:49.029739 836 raft_consensus.cc:686] T 5c6ed28a34454f57b41b889ba884d304 P 6d7acbb7aab64a218850d78341c533f9 [term 1 LEADER]: Becoming Leader. State: Replica: 6d7acbb7aab64a218850d78341c533f9, State: Running, Role: LEADER
I20260811 05:12:49.029882 836 consensus_queue.cc:237] T 5c6ed28a34454f57b41b889ba884d304 P 6d7acbb7aab64a218850d78341c533f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } }
I20260811 05:12:49.033247 689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c6ed28a34454f57b41b889ba884d304" candidate_uuid: "6d7acbb7aab64a218850d78341c533f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35b5b28f83c408ab44e03ffd27c81e4"
I20260811 05:12:49.033411 689 raft_consensus.cc:3037] T 5c6ed28a34454f57b41b889ba884d304 P d35b5b28f83c408ab44e03ffd27c81e4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:49.033959 32377 catalog_manager.cc:5495] T 5c6ed28a34454f57b41b889ba884d304 P 6d7acbb7aab64a218850d78341c533f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d7acbb7aab64a218850d78341c533f9 (127.28.143.129). New cstate: current_term: 1 leader_uuid: "6d7acbb7aab64a218850d78341c533f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } health_report { overall_health: UNKNOWN } } }
I20260811 05:12:49.034505 689 raft_consensus.cc:2454] T 5c6ed28a34454f57b41b889ba884d304 P d35b5b28f83c408ab44e03ffd27c81e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d7acbb7aab64a218850d78341c533f9 in term 1.
Aug 11 05:12:49 dist-test-slave-f7th krb5kdc[32291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425168, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:12:49.089181 32468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:49.081605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54206 (local address 127.28.143.190:44957)
0811 05:12:49.081675 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:49.081677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:49.081802 (+ 125us) server_negotiation.cc:407] Connection header received
0811 05:12:49.081831 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:49.081832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:49.081841 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:49.081895 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:49.082355 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:49.082942 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:49.083796 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:49.084003 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:49.084066 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:49.086320 (+ 2254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:49.086321 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:49.086321 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:49.086330 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:49.087737 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:49.088211 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:49.088212 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:49.088213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:49.088274 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:49.088435 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:49.088435 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:49.088436 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:49.088547 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:49.088597 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:49.088953 (+ 356us) server_negotiation.cc:299] Negotiation successful
0811 05:12:49.089094 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":40}
I20260811 05:12:49.089182 853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:49.081525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:44957 (local address 127.0.0.1:54206)
0811 05:12:49.081711 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:49.081722 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:49.081793 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:49.081906 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:49.081907 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:49.082213 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:49.082261 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:49.082263 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:49.083097 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:49.083098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:49.083712 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:49.083714 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:49.084078 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:49.084079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:49.084128 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:49.084751 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:12:49.084756 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:49.086209 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:49.087849 (+ 1640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:49.087858 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:49.087861 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:49.088120 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:49.088336 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:49.088336 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:49.088336 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:49.088391 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:49.088747 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:49.088756 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:49.088813 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 05:12:49.088881 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 05:12:49.089032 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":87,"threads_started":1}
I20260811 05:12:49.091420 32377 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:54206:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 05:12:49.091624 32377 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:12:49.093264 32628 tablet_service.cc:1467] Processing CreateTablet for tablet 4d0ee3d051a342528132b6d31a0b0260 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f860353fb72347c186bf0edc19f46175]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 05:12:49.093292 412 tablet_service.cc:1467] Processing CreateTablet for tablet 4d0ee3d051a342528132b6d31a0b0260 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f860353fb72347c186bf0edc19f46175]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 05:12:49.093353 649 tablet_service.cc:1467] Processing CreateTablet for tablet 4d0ee3d051a342528132b6d31a0b0260 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f860353fb72347c186bf0edc19f46175]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 05:12:49.093483 32628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d0ee3d051a342528132b6d31a0b0260. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:49.093482 412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d0ee3d051a342528132b6d31a0b0260. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:49.093538 649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d0ee3d051a342528132b6d31a0b0260. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:49.094945 821 tablet_bootstrap.cc:492] T 4d0ee3d051a342528132b6d31a0b0260 P d35b5b28f83c408ab44e03ffd27c81e4: Bootstrap starting.
I20260811 05:12:49.095034 822 tablet_bootstrap.cc:492] T 4d0ee3d051a342528132b6d31a0b0260 P 6d7acbb7aab64a218850d78341c533f9: Bootstrap starting.
I20260811 05:12:49.095472 822 tablet_bootstrap.cc:654] T 4d0ee3d051a342528132b6d31a0b0260 P 6d7acbb7aab64a218850d78341c533f9: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:49.095605 821 tablet_bootstrap.cc:654] T 4d0ee3d051a342528132b6d31a0b0260 P d35b5b28f83c408ab44e03ffd27c81e4: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:49.096040 819 tablet_bootstrap.cc:492] T 4d0ee3d051a342528132b6d31a0b0260 P c80c0a0fe54e4382abe4e7a0345fe9f5: Bootstrap starting.
I20260811 05:12:49.096520 821 tablet_bootstrap.cc:492] T 4d0ee3d051a342528132b6d31a0b0260 P d35b5b28f83c408ab44e03ffd27c81e4: No bootstrap required, opened a new log
I20260811 05:12:49.096573 821 ts_tablet_manager.cc:1397] T 4d0ee3d051a342528132b6d31a0b0260 P d35b5b28f83c408ab44e03ffd27c81e4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260811 05:12:49.096719 822 tablet_bootstrap.cc:492] T 4d0ee3d051a342528132b6d31a0b0260 P 6d7acbb7aab64a218850d78341c533f9: No bootstrap required, opened a new log
I20260811 05:12:49.096763 822 ts_tablet_manager.cc:1397] T 4d0ee3d051a342528132b6d31a0b0260 P 6d7acbb7aab64a218850d78341c533f9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 05:12:49.096751 821 raft_consensus.cc:348] T 4d0ee3d051a342528132b6d31a0b0260 P d35b5b28f83c408ab44e03ffd27c81e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } }
I20260811 05:12:49.096801 819 tablet_bootstrap.cc:654] T 4d0ee3d051a342528132b6d31a0b0260 P c80c0a0fe54e4382abe4e7a0345fe9f5: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:49.096819 821 raft_consensus.cc:374] T 4d0ee3d051a342528132b6d31a0b0260 P d35b5b28f83c408ab44e03ffd27c81e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:49.096833 821 raft_consensus.cc:729] T 4d0ee3d051a342528132b6d31a0b0260 P d35b5b28f83c408ab44e03ffd27c81e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35b5b28f83c408ab44e03ffd27c81e4, State: Initialized, Role: FOLLOWER
I20260811 05:12:49.096874 821 consensus_queue.cc:260] T 4d0ee3d051a342528132b6d31a0b0260 P d35b5b28f83c408ab44e03ffd27c81e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } }
I20260811 05:12:49.096930 822 raft_consensus.cc:348] T 4d0ee3d051a342528132b6d31a0b0260 P 6d7acbb7aab64a218850d78341c533f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } }
I20260811 05:12:49.096976 821 ts_tablet_manager.cc:1428] T 4d0ee3d051a342528132b6d31a0b0260 P d35b5b28f83c408ab44e03ffd27c81e4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:49.097004 822 raft_consensus.cc:374] T 4d0ee3d051a342528132b6d31a0b0260 P 6d7acbb7aab64a218850d78341c533f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:49.097025 822 raft_consensus.cc:729] T 4d0ee3d051a342528132b6d31a0b0260 P 6d7acbb7aab64a218850d78341c533f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d7acbb7aab64a218850d78341c533f9, State: Initialized, Role: FOLLOWER
I20260811 05:12:49.097074 822 consensus_queue.cc:260] T 4d0ee3d051a342528132b6d31a0b0260 P 6d7acbb7aab64a218850d78341c533f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } }
I20260811 05:12:49.097069 837 tablet_replica.cc:439] T 4d0ee3d051a342528132b6d31a0b0260 P d35b5b28f83c408ab44e03ffd27c81e4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } } }
I20260811 05:12:49.097162 822 ts_tablet_manager.cc:1428] T 4d0ee3d051a342528132b6d31a0b0260 P 6d7acbb7aab64a218850d78341c533f9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:49.097168 837 tablet_replica.cc:442] T 4d0ee3d051a342528132b6d31a0b0260 P d35b5b28f83c408ab44e03ffd27c81e4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:12:49.097237 836 tablet_replica.cc:439] T 4d0ee3d051a342528132b6d31a0b0260 P 6d7acbb7aab64a218850d78341c533f9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } } }
I20260811 05:12:49.097325 836 tablet_replica.cc:442] T 4d0ee3d051a342528132b6d31a0b0260 P 6d7acbb7aab64a218850d78341c533f9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:12:49.098030 819 tablet_bootstrap.cc:492] T 4d0ee3d051a342528132b6d31a0b0260 P c80c0a0fe54e4382abe4e7a0345fe9f5: No bootstrap required, opened a new log
I20260811 05:12:49.098091 819 ts_tablet_manager.cc:1397] T 4d0ee3d051a342528132b6d31a0b0260 P c80c0a0fe54e4382abe4e7a0345fe9f5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 05:12:49.098246 819 raft_consensus.cc:348] T 4d0ee3d051a342528132b6d31a0b0260 P c80c0a0fe54e4382abe4e7a0345fe9f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } }
I20260811 05:12:49.098312 819 raft_consensus.cc:374] T 4d0ee3d051a342528132b6d31a0b0260 P c80c0a0fe54e4382abe4e7a0345fe9f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:49.098326 819 raft_consensus.cc:729] T 4d0ee3d051a342528132b6d31a0b0260 P c80c0a0fe54e4382abe4e7a0345fe9f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c80c0a0fe54e4382abe4e7a0345fe9f5, State: Initialized, Role: FOLLOWER
I20260811 05:12:49.098382 819 consensus_queue.cc:260] T 4d0ee3d051a342528132b6d31a0b0260 P c80c0a0fe54e4382abe4e7a0345fe9f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } }
I20260811 05:12:49.098470 819 ts_tablet_manager.cc:1428] T 4d0ee3d051a342528132b6d31a0b0260 P c80c0a0fe54e4382abe4e7a0345fe9f5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:49.098568 834 tablet_replica.cc:439] T 4d0ee3d051a342528132b6d31a0b0260 P c80c0a0fe54e4382abe4e7a0345fe9f5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } } }
I20260811 05:12:49.098649 834 tablet_replica.cc:442] T 4d0ee3d051a342528132b6d31a0b0260 P c80c0a0fe54e4382abe4e7a0345fe9f5: This TxnStatusTablet replica's current role is: FOLLOWER
W20260811 05:12:49.106521 782 tablet.cc:2369] T 4d0ee3d051a342528132b6d31a0b0260 P d35b5b28f83c408ab44e03ffd27c81e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:12:49.135383 837 raft_consensus.cc:482] T 4d0ee3d051a342528132b6d31a0b0260 P d35b5b28f83c408ab44e03ffd27c81e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:12:49.135509 837 raft_consensus.cc:504] T 4d0ee3d051a342528132b6d31a0b0260 P d35b5b28f83c408ab44e03ffd27c81e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } }
I20260811 05:12:49.135841 837 leader_election.cc:288] T 4d0ee3d051a342528132b6d31a0b0260 P d35b5b28f83c408ab44e03ffd27c81e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d7acbb7aab64a218850d78341c533f9 (127.28.143.129:43067), c80c0a0fe54e4382abe4e7a0345fe9f5 (127.28.143.130:37639)
I20260811 05:12:49.139178 817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:49.136151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:43961 (local address 127.28.143.129:43067)
0811 05:12:49.136218 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:49.136220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:49.136306 (+ 86us) server_negotiation.cc:407] Connection header received
0811 05:12:49.136434 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:49.136435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:49.136446 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:49.136499 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:49.136731 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:49.137228 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:49.138201 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:49.138716 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:49.138820 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:49.138831 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:49.139056 (+ 225us) server_negotiation.cc:299] Negotiation successful
0811 05:12:49.139101 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 05:12:49.139190 580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:49.136066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:43067 (local address 127.28.143.131:43961)
0811 05:12:49.136210 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:49.136224 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:49.136304 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:49.136553 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:49.136555 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:49.136558 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:49.136665 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:49.136668 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:49.137314 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:49.137316 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:49.138103 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:49.138106 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:49.138856 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:49.138858 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:49.138941 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:49.138945 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:12:49.139048 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 05:12:49.139096 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118}
I20260811 05:12:49.139230 813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:49.136301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:55725 (local address 127.28.143.130:37639)
0811 05:12:49.136346 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:49.136347 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:49.136500 (+ 153us) server_negotiation.cc:407] Connection header received
0811 05:12:49.136541 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:49.136543 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:49.136554 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:49.136659 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:49.136879 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:49.137466 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:49.138388 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:49.138905 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:49.138959 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:49.138965 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:49.139112 (+ 147us) server_negotiation.cc:299] Negotiation successful
0811 05:12:49.139171 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 05:12:49.139204 864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:49.136179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:37639 (local address 127.28.143.131:55725)
0811 05:12:49.136382 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:49.136392 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:49.136459 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:49.136721 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:49.136723 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:49.136725 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:49.136798 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:49.136800 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:49.137580 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:49.137581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:49.138298 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:49.138300 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:49.138982 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:49.138984 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:49.139044 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:49.139048 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:12:49.139104 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 05:12:49.139138 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":133,"threads_started":1}
I20260811 05:12:49.139596 32654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d0ee3d051a342528132b6d31a0b0260" candidate_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d7acbb7aab64a218850d78341c533f9" is_pre_election: true
I20260811 05:12:49.139744 32654 raft_consensus.cc:2454] T 4d0ee3d051a342528132b6d31a0b0260 P 6d7acbb7aab64a218850d78341c533f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35b5b28f83c408ab44e03ffd27c81e4 in term 0.
I20260811 05:12:49.139704 449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d0ee3d051a342528132b6d31a0b0260" candidate_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" is_pre_election: true
I20260811 05:12:49.139802 449 raft_consensus.cc:2454] T 4d0ee3d051a342528132b6d31a0b0260 P c80c0a0fe54e4382abe4e7a0345fe9f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35b5b28f83c408ab44e03ffd27c81e4 in term 0.
I20260811 05:12:49.140002 574 leader_election.cc:302] T 4d0ee3d051a342528132b6d31a0b0260 P d35b5b28f83c408ab44e03ffd27c81e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c80c0a0fe54e4382abe4e7a0345fe9f5, d35b5b28f83c408ab44e03ffd27c81e4; no voters:
I20260811 05:12:49.140147 837 raft_consensus.cc:2781] T 4d0ee3d051a342528132b6d31a0b0260 P d35b5b28f83c408ab44e03ffd27c81e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:12:49.140196 837 raft_consensus.cc:482] T 4d0ee3d051a342528132b6d31a0b0260 P d35b5b28f83c408ab44e03ffd27c81e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:12:49.140223 837 raft_consensus.cc:3037] T 4d0ee3d051a342528132b6d31a0b0260 P d35b5b28f83c408ab44e03ffd27c81e4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:49.141115 837 raft_consensus.cc:504] T 4d0ee3d051a342528132b6d31a0b0260 P d35b5b28f83c408ab44e03ffd27c81e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } }
I20260811 05:12:49.141254 837 leader_election.cc:288] T 4d0ee3d051a342528132b6d31a0b0260 P d35b5b28f83c408ab44e03ffd27c81e4 [CANDIDATE]: Term 1 election: Requested vote from peers 6d7acbb7aab64a218850d78341c533f9 (127.28.143.129:43067), c80c0a0fe54e4382abe4e7a0345fe9f5 (127.28.143.130:37639)
I20260811 05:12:49.141474 32654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d0ee3d051a342528132b6d31a0b0260" candidate_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d7acbb7aab64a218850d78341c533f9"
I20260811 05:12:49.141487 449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d0ee3d051a342528132b6d31a0b0260" candidate_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5"
I20260811 05:12:49.141557 32654 raft_consensus.cc:3037] T 4d0ee3d051a342528132b6d31a0b0260 P 6d7acbb7aab64a218850d78341c533f9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:49.141566 449 raft_consensus.cc:3037] T 4d0ee3d051a342528132b6d31a0b0260 P c80c0a0fe54e4382abe4e7a0345fe9f5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:49.142256 449 raft_consensus.cc:2454] T 4d0ee3d051a342528132b6d31a0b0260 P c80c0a0fe54e4382abe4e7a0345fe9f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35b5b28f83c408ab44e03ffd27c81e4 in term 1.
I20260811 05:12:49.142264 32654 raft_consensus.cc:2454] T 4d0ee3d051a342528132b6d31a0b0260 P 6d7acbb7aab64a218850d78341c533f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35b5b28f83c408ab44e03ffd27c81e4 in term 1.
I20260811 05:12:49.142470 574 leader_election.cc:302] T 4d0ee3d051a342528132b6d31a0b0260 P d35b5b28f83c408ab44e03ffd27c81e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c80c0a0fe54e4382abe4e7a0345fe9f5, d35b5b28f83c408ab44e03ffd27c81e4; no voters:
I20260811 05:12:49.142575 837 raft_consensus.cc:2781] T 4d0ee3d051a342528132b6d31a0b0260 P d35b5b28f83c408ab44e03ffd27c81e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:49.142753 837 raft_consensus.cc:686] T 4d0ee3d051a342528132b6d31a0b0260 P d35b5b28f83c408ab44e03ffd27c81e4 [term 1 LEADER]: Becoming Leader. State: Replica: d35b5b28f83c408ab44e03ffd27c81e4, State: Running, Role: LEADER
I20260811 05:12:49.142848 837 consensus_queue.cc:237] T 4d0ee3d051a342528132b6d31a0b0260 P d35b5b28f83c408ab44e03ffd27c81e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } }
I20260811 05:12:49.143316 865 tablet_replica.cc:439] T 4d0ee3d051a342528132b6d31a0b0260 P d35b5b28f83c408ab44e03ffd27c81e4: TxnStatusTablet state changed. Reason: New leader d35b5b28f83c408ab44e03ffd27c81e4. Latest consensus state: current_term: 1 leader_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } } }
I20260811 05:12:49.143393 865 tablet_replica.cc:442] T 4d0ee3d051a342528132b6d31a0b0260 P d35b5b28f83c408ab44e03ffd27c81e4: This TxnStatusTablet replica's current role is: LEADER
I20260811 05:12:49.143561 871 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 05:12:49.143678 871 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 05:12:49.143925 32377 catalog_manager.cc:5495] T 4d0ee3d051a342528132b6d31a0b0260 P d35b5b28f83c408ab44e03ffd27c81e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to d35b5b28f83c408ab44e03ffd27c81e4 (127.28.143.131). New cstate: current_term: 1 leader_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } health_report { overall_health: HEALTHY } } }
I20260811 05:12:49.158828 853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:49.155744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.131:38799 (local address 127.0.0.1:33684)
0811 05:12:49.155896 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:49.155911 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:49.155970 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:49.156182 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:49.156183 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:49.156185 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 05:12:49.156238 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:49.156241 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:49.157201 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:49.157203 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:49.157802 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:49.157805 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:49.158273 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:49.158274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:49.158332 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:49.158355 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 05:12:49.158655 (+ 300us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 05:12:49.158664 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 05:12:49.158705 (+ 41us) client_negotiation.cc:240] Negotiation successful
0811 05:12:49.158761 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 05:12:49.158847 818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:49.155844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33684 (local address 127.28.143.131:38799)
0811 05:12:49.155958 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:49.155961 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:49.155972 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:12:49.156044 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:49.156045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:49.156067 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:49.156139 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:12:49.156309 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:49.157098 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:49.157888 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:49.158190 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:49.158260 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:49.158447 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:12:49.158590 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:12:49.158670 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:49.158712 (+ 42us) server_negotiation.cc:299] Negotiation successful
0811 05:12:49.158735 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 05:12:49.160956 32654 raft_consensus.cc:1261] T 4d0ee3d051a342528132b6d31a0b0260 P 6d7acbb7aab64a218850d78341c533f9 [term 1 FOLLOWER]: Refusing update from remote peer d35b5b28f83c408ab44e03ffd27c81e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:12:49.161016 449 raft_consensus.cc:1261] T 4d0ee3d051a342528132b6d31a0b0260 P c80c0a0fe54e4382abe4e7a0345fe9f5 [term 1 FOLLOWER]: Refusing update from remote peer d35b5b28f83c408ab44e03ffd27c81e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:12:49.161316 865 consensus_queue.cc:1035] T 4d0ee3d051a342528132b6d31a0b0260 P d35b5b28f83c408ab44e03ffd27c81e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:12:49.161477 837 consensus_queue.cc:1035] T 4d0ee3d051a342528132b6d31a0b0260 P d35b5b28f83c408ab44e03ffd27c81e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:12:49.162781 834 tablet_replica.cc:439] T 4d0ee3d051a342528132b6d31a0b0260 P c80c0a0fe54e4382abe4e7a0345fe9f5: TxnStatusTablet state changed. Reason: New leader d35b5b28f83c408ab44e03ffd27c81e4. Latest consensus state: current_term: 1 leader_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } } }
I20260811 05:12:49.162783 836 tablet_replica.cc:439] T 4d0ee3d051a342528132b6d31a0b0260 P 6d7acbb7aab64a218850d78341c533f9: TxnStatusTablet state changed. Reason: New leader d35b5b28f83c408ab44e03ffd27c81e4. Latest consensus state: current_term: 1 leader_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } } }
I20260811 05:12:49.162887 836 tablet_replica.cc:442] T 4d0ee3d051a342528132b6d31a0b0260 P 6d7acbb7aab64a218850d78341c533f9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:12:49.162889 834 tablet_replica.cc:442] T 4d0ee3d051a342528132b6d31a0b0260 P c80c0a0fe54e4382abe4e7a0345fe9f5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:12:49.164126 837 tablet_replica.cc:439] T 4d0ee3d051a342528132b6d31a0b0260 P d35b5b28f83c408ab44e03ffd27c81e4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } } }
I20260811 05:12:49.164223 837 tablet_replica.cc:442] T 4d0ee3d051a342528132b6d31a0b0260 P d35b5b28f83c408ab44e03ffd27c81e4: This TxnStatusTablet replica's current role is: LEADER
I20260811 05:12:49.164266 837 tablet_replica.cc:439] T 4d0ee3d051a342528132b6d31a0b0260 P d35b5b28f83c408ab44e03ffd27c81e4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } } }
I20260811 05:12:49.164302 837 tablet_replica.cc:442] T 4d0ee3d051a342528132b6d31a0b0260 P d35b5b28f83c408ab44e03ffd27c81e4: This TxnStatusTablet replica's current role is: LEADER
I20260811 05:12:49.164461 836 tablet_replica.cc:439] T 4d0ee3d051a342528132b6d31a0b0260 P 6d7acbb7aab64a218850d78341c533f9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } } }
I20260811 05:12:49.164543 836 tablet_replica.cc:442] T 4d0ee3d051a342528132b6d31a0b0260 P 6d7acbb7aab64a218850d78341c533f9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:12:49.164494 834 tablet_replica.cc:439] T 4d0ee3d051a342528132b6d31a0b0260 P c80c0a0fe54e4382abe4e7a0345fe9f5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } } }
I20260811 05:12:49.164575 834 tablet_replica.cc:442] T 4d0ee3d051a342528132b6d31a0b0260 P c80c0a0fe54e4382abe4e7a0345fe9f5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:12:49.164670 880 mvcc.cc:205] Tried to move back new op lower bound from 7317197492880019456 to 7317197492809695232. Current Snapshot: MvccSnapshot[applied={T|T < 7317197492880019456}]
I20260811 05:12:49.164882 882 mvcc.cc:205] Tried to move back new op lower bound from 7317197492880019456 to 7317197492809695232. Current Snapshot: MvccSnapshot[applied={T|T < 7317197492880019456}]
I20260811 05:12:49.165566 881 mvcc.cc:205] Tried to move back new op lower bound from 7317197492880019456 to 7317197492809695232. Current Snapshot: MvccSnapshot[applied={T|T < 7317197492880019456 or (T in {7317197492880019456})}]
I20260811 05:12:49.170526 817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:49.167726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45940 (local address 127.28.143.129:43067)
0811 05:12:49.167765 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:49.167767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:49.167778 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:12:49.167863 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:49.167864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:49.167880 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:49.167939 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:12:49.168071 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:49.168543 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:49.169706 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:49.169957 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:49.170050 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:49.170205 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:12:49.170329 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:12:49.170384 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:49.170441 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 05:12:49.170462 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 05:12:49.175518 818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:49.173165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:36279 (local address 127.28.143.131:38799)
0811 05:12:49.173216 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:49.173218 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:49.173307 (+ 89us) server_negotiation.cc:407] Connection header received
0811 05:12:49.173362 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:49.173363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:49.173373 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:49.173418 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:12:49.173560 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:49.174021 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:49.174831 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:49.175019 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:49.175078 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:49.175271 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:12:49.175302 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:12:49.175343 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:49.175403 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 05:12:49.175425 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 05:12:49.175532 896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:49.173085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.131:38799 (local address 127.28.143.129:36279)
0811 05:12:49.173240 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:49.173251 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:49.173310 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:49.173419 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:49.173419 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:49.173421 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 05:12:49.173489 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:49.173490 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:49.174096 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:49.174097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:49.174692 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:49.174694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:49.175089 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:49.175090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:49.175158 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:49.175176 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 05:12:49.175352 (+ 176us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 05:12:49.175354 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 05:12:49.175391 (+ 37us) client_negotiation.cc:240] Negotiation successful
0811 05:12:49.175438 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":81,"threads_started":1}
I20260811 05:12:49.178323 449 raft_consensus.cc:1261] T 5c6ed28a34454f57b41b889ba884d304 P c80c0a0fe54e4382abe4e7a0345fe9f5 [term 1 FOLLOWER]: Refusing update from remote peer 6d7acbb7aab64a218850d78341c533f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:12:49.178402 689 raft_consensus.cc:1261] T 5c6ed28a34454f57b41b889ba884d304 P d35b5b28f83c408ab44e03ffd27c81e4 [term 1 FOLLOWER]: Refusing update from remote peer 6d7acbb7aab64a218850d78341c533f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:12:49.178606 840 consensus_queue.cc:1035] T 5c6ed28a34454f57b41b889ba884d304 P 6d7acbb7aab64a218850d78341c533f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:12:49.178747 836 consensus_queue.cc:1035] T 5c6ed28a34454f57b41b889ba884d304 P 6d7acbb7aab64a218850d78341c533f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:12:49.189072 817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:49.186274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:52179 (local address 127.28.143.129:43067)
0811 05:12:49.186319 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:49.186321 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:49.186328 (+ 7us) server_negotiation.cc:407] Connection header received
0811 05:12:49.186346 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:49.186347 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:49.186355 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:49.186397 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:12:49.186575 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:49.187067 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:49.188177 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:49.188501 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:49.188566 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:49.188805 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:12:49.188846 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:12:49.188898 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:49.188961 (+ 63us) server_negotiation.cc:299] Negotiation successful
0811 05:12:49.188988 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 05:12:49.189093 792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:49.186080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:43067 (local address 127.28.143.131:52179)
0811 05:12:49.186147 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:49.186159 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:49.186200 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:49.186422 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:49.186423 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:49.186424 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 05:12:49.186514 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:49.186516 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:49.187142 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:49.187143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:49.188073 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:49.188076 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:49.188576 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:49.188578 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:49.188685 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:49.188710 (+ 25us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 05:12:49.188906 (+ 196us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 05:12:49.188909 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 05:12:49.188955 (+ 46us) client_negotiation.cc:240] Negotiation successful
0811 05:12:49.189001 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260811 05:12:49.199541 32377 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:12:49.199710 32377 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:12:49.200759 32377 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 251ba2f17a774486bcc0fa8cfea5c90f: Sending DeleteTablet for 3 replicas of tablet 5c6ed28a34454f57b41b889ba884d304
I20260811 05:12:49.201172 649 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c6ed28a34454f57b41b889ba884d304 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:12:49 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:33680
I20260811 05:12:49.201265 32628 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c6ed28a34454f57b41b889ba884d304 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:12:49 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:45924
I20260811 05:12:49.201793 412 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c6ed28a34454f57b41b889ba884d304 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:12:49 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:44154
I20260811 05:12:49.201875 32377 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54204:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 05:12:49.202033 916 tablet_replica.cc:330] T 5c6ed28a34454f57b41b889ba884d304 P 6d7acbb7aab64a218850d78341c533f9: stopping tablet replica
W20260811 05:12:49.202049 32377 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:12:49.202147 916 raft_consensus.cc:2229] T 5c6ed28a34454f57b41b889ba884d304 P 6d7acbb7aab64a218850d78341c533f9 [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:12:49.202255 916 raft_consensus.cc:2258] T 5c6ed28a34454f57b41b889ba884d304 P 6d7acbb7aab64a218850d78341c533f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:12:49.202404 917 tablet_replica.cc:330] T 5c6ed28a34454f57b41b889ba884d304 P c80c0a0fe54e4382abe4e7a0345fe9f5: stopping tablet replica
I20260811 05:12:49.202522 917 raft_consensus.cc:2229] T 5c6ed28a34454f57b41b889ba884d304 P c80c0a0fe54e4382abe4e7a0345fe9f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:12:49.202597 917 raft_consensus.cc:2258] T 5c6ed28a34454f57b41b889ba884d304 P c80c0a0fe54e4382abe4e7a0345fe9f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:12:49.202767 914 tablet_replica.cc:330] T 5c6ed28a34454f57b41b889ba884d304 P d35b5b28f83c408ab44e03ffd27c81e4: stopping tablet replica
I20260811 05:12:49.202879 914 raft_consensus.cc:2229] T 5c6ed28a34454f57b41b889ba884d304 P d35b5b28f83c408ab44e03ffd27c81e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:12:49.202955 914 raft_consensus.cc:2258] T 5c6ed28a34454f57b41b889ba884d304 P d35b5b28f83c408ab44e03ffd27c81e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:12:49.203105 917 ts_tablet_manager.cc:1905] T 5c6ed28a34454f57b41b889ba884d304 P c80c0a0fe54e4382abe4e7a0345fe9f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:12:49.203398 914 ts_tablet_manager.cc:1905] T 5c6ed28a34454f57b41b889ba884d304 P d35b5b28f83c408ab44e03ffd27c81e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:12:49.204176 32628 tablet_service.cc:1467] Processing CreateTablet for tablet 6c6d37788be1427da6d04d4f2675ee7b (DEFAULT_TABLE table=test-table [id=a75a843a4fef4af3b8941889c05d26c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:12:49.204313 32628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c6d37788be1427da6d04d4f2675ee7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:49.205104 914 ts_tablet_manager.cc:1918] T 5c6ed28a34454f57b41b889ba884d304 P d35b5b28f83c408ab44e03ffd27c81e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 05:12:49.205142 916 ts_tablet_manager.cc:1905] T 5c6ed28a34454f57b41b889ba884d304 P 6d7acbb7aab64a218850d78341c533f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:12:49.205178 914 log.cc:1196] T 5c6ed28a34454f57b41b889ba884d304 P d35b5b28f83c408ab44e03ffd27c81e4: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-2/wal/wals/5c6ed28a34454f57b41b889ba884d304
I20260811 05:12:49.205441 914 ts_tablet_manager.cc:1939] T 5c6ed28a34454f57b41b889ba884d304 P d35b5b28f83c408ab44e03ffd27c81e4: Deleting consensus metadata
I20260811 05:12:49.205821 412 tablet_service.cc:1467] Processing CreateTablet for tablet 6c6d37788be1427da6d04d4f2675ee7b (DEFAULT_TABLE table=test-table [id=a75a843a4fef4af3b8941889c05d26c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:12:49.205945 822 tablet_bootstrap.cc:492] T 6c6d37788be1427da6d04d4f2675ee7b P 6d7acbb7aab64a218850d78341c533f9: Bootstrap starting.
I20260811 05:12:49.205986 32347 catalog_manager.cc:4841] TS d35b5b28f83c408ab44e03ffd27c81e4 (127.28.143.131:38799): tablet 5c6ed28a34454f57b41b889ba884d304 (table test-table [id=7fe4524343b04bf8a69a3d13f0e19365]) successfully deleted
I20260811 05:12:49.206384 822 tablet_bootstrap.cc:654] T 6c6d37788be1427da6d04d4f2675ee7b P 6d7acbb7aab64a218850d78341c533f9: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:49.207041 822 tablet_bootstrap.cc:492] T 6c6d37788be1427da6d04d4f2675ee7b P 6d7acbb7aab64a218850d78341c533f9: No bootstrap required, opened a new log
I20260811 05:12:49.207098 822 ts_tablet_manager.cc:1397] T 6c6d37788be1427da6d04d4f2675ee7b P 6d7acbb7aab64a218850d78341c533f9: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260811 05:12:49.207356 822 raft_consensus.cc:348] T 6c6d37788be1427da6d04d4f2675ee7b P 6d7acbb7aab64a218850d78341c533f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } }
I20260811 05:12:49.207444 822 raft_consensus.cc:374] T 6c6d37788be1427da6d04d4f2675ee7b P 6d7acbb7aab64a218850d78341c533f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:49.207466 822 raft_consensus.cc:729] T 6c6d37788be1427da6d04d4f2675ee7b P 6d7acbb7aab64a218850d78341c533f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d7acbb7aab64a218850d78341c533f9, State: Initialized, Role: FOLLOWER
I20260811 05:12:49.207508 822 consensus_queue.cc:260] T 6c6d37788be1427da6d04d4f2675ee7b P 6d7acbb7aab64a218850d78341c533f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } }
I20260811 05:12:49.207497 649 tablet_service.cc:1467] Processing CreateTablet for tablet 6c6d37788be1427da6d04d4f2675ee7b (DEFAULT_TABLE table=test-table [id=a75a843a4fef4af3b8941889c05d26c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:12:49.207602 822 ts_tablet_manager.cc:1428] T 6c6d37788be1427da6d04d4f2675ee7b P 6d7acbb7aab64a218850d78341c533f9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:49.207623 649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c6d37788be1427da6d04d4f2675ee7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:49.208905 821 tablet_bootstrap.cc:492] T 6c6d37788be1427da6d04d4f2675ee7b P d35b5b28f83c408ab44e03ffd27c81e4: Bootstrap starting.
I20260811 05:12:49.209419 821 tablet_bootstrap.cc:654] T 6c6d37788be1427da6d04d4f2675ee7b P d35b5b28f83c408ab44e03ffd27c81e4: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:49.210022 821 tablet_bootstrap.cc:492] T 6c6d37788be1427da6d04d4f2675ee7b P d35b5b28f83c408ab44e03ffd27c81e4: No bootstrap required, opened a new log
I20260811 05:12:49.210081 821 ts_tablet_manager.cc:1397] T 6c6d37788be1427da6d04d4f2675ee7b P d35b5b28f83c408ab44e03ffd27c81e4: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260811 05:12:49.210240 821 raft_consensus.cc:348] T 6c6d37788be1427da6d04d4f2675ee7b P d35b5b28f83c408ab44e03ffd27c81e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } }
I20260811 05:12:49.210318 821 raft_consensus.cc:374] T 6c6d37788be1427da6d04d4f2675ee7b P d35b5b28f83c408ab44e03ffd27c81e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:49.210338 821 raft_consensus.cc:729] T 6c6d37788be1427da6d04d4f2675ee7b P d35b5b28f83c408ab44e03ffd27c81e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35b5b28f83c408ab44e03ffd27c81e4, State: Initialized, Role: FOLLOWER
I20260811 05:12:49.210399 821 consensus_queue.cc:260] T 6c6d37788be1427da6d04d4f2675ee7b P d35b5b28f83c408ab44e03ffd27c81e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } }
I20260811 05:12:49.210494 821 ts_tablet_manager.cc:1428] T 6c6d37788be1427da6d04d4f2675ee7b P d35b5b28f83c408ab44e03ffd27c81e4: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260811 05:12:49.214988 840 raft_consensus.cc:482] T 6c6d37788be1427da6d04d4f2675ee7b P 6d7acbb7aab64a218850d78341c533f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:12:49.215116 840 raft_consensus.cc:504] T 6c6d37788be1427da6d04d4f2675ee7b P 6d7acbb7aab64a218850d78341c533f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } }
I20260811 05:12:49.215296 840 leader_election.cc:288] T 6c6d37788be1427da6d04d4f2675ee7b P 6d7acbb7aab64a218850d78341c533f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c80c0a0fe54e4382abe4e7a0345fe9f5 (127.28.143.130:37639), d35b5b28f83c408ab44e03ffd27c81e4 (127.28.143.131:38799)
I20260811 05:12:49.215485 689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c6d37788be1427da6d04d4f2675ee7b" candidate_uuid: "6d7acbb7aab64a218850d78341c533f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" is_pre_election: true
I20260811 05:12:49.215584 689 raft_consensus.cc:2454] T 6c6d37788be1427da6d04d4f2675ee7b P d35b5b28f83c408ab44e03ffd27c81e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d7acbb7aab64a218850d78341c533f9 in term 0.
I20260811 05:12:49.215745 32534 leader_election.cc:302] T 6c6d37788be1427da6d04d4f2675ee7b P 6d7acbb7aab64a218850d78341c533f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d7acbb7aab64a218850d78341c533f9, d35b5b28f83c408ab44e03ffd27c81e4; no voters:
W20260811 05:12:49.215817 32536 leader_election.cc:341] T 6c6d37788be1427da6d04d4f2675ee7b P 6d7acbb7aab64a218850d78341c533f9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c80c0a0fe54e4382abe4e7a0345fe9f5 (127.28.143.130:37639): Not found: Tablet not found: 6c6d37788be1427da6d04d4f2675ee7b
I20260811 05:12:49.215829 840 raft_consensus.cc:2781] T 6c6d37788be1427da6d04d4f2675ee7b P 6d7acbb7aab64a218850d78341c533f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:12:49.216025 840 raft_consensus.cc:482] T 6c6d37788be1427da6d04d4f2675ee7b P 6d7acbb7aab64a218850d78341c533f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:12:49.216046 840 raft_consensus.cc:3037] T 6c6d37788be1427da6d04d4f2675ee7b P 6d7acbb7aab64a218850d78341c533f9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:49.216801 917 ts_tablet_manager.cc:1918] T 5c6ed28a34454f57b41b889ba884d304 P c80c0a0fe54e4382abe4e7a0345fe9f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 05:12:49.216802 412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c6d37788be1427da6d04d4f2675ee7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:49.216918 917 log.cc:1196] T 5c6ed28a34454f57b41b889ba884d304 P c80c0a0fe54e4382abe4e7a0345fe9f5: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-1/wal/wals/5c6ed28a34454f57b41b889ba884d304
I20260811 05:12:49.216993 840 raft_consensus.cc:504] T 6c6d37788be1427da6d04d4f2675ee7b P 6d7acbb7aab64a218850d78341c533f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } }
I20260811 05:12:49.217149 840 leader_election.cc:288] T 6c6d37788be1427da6d04d4f2675ee7b P 6d7acbb7aab64a218850d78341c533f9 [CANDIDATE]: Term 1 election: Requested vote from peers c80c0a0fe54e4382abe4e7a0345fe9f5 (127.28.143.130:37639), d35b5b28f83c408ab44e03ffd27c81e4 (127.28.143.131:38799)
I20260811 05:12:49.217218 917 ts_tablet_manager.cc:1939] T 5c6ed28a34454f57b41b889ba884d304 P c80c0a0fe54e4382abe4e7a0345fe9f5: Deleting consensus metadata
W20260811 05:12:49.217482 32536 leader_election.cc:341] T 6c6d37788be1427da6d04d4f2675ee7b P 6d7acbb7aab64a218850d78341c533f9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c80c0a0fe54e4382abe4e7a0345fe9f5 (127.28.143.130:37639): Not found: Tablet not found: 6c6d37788be1427da6d04d4f2675ee7b
I20260811 05:12:49.217665 689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c6d37788be1427da6d04d4f2675ee7b" candidate_uuid: "6d7acbb7aab64a218850d78341c533f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35b5b28f83c408ab44e03ffd27c81e4"
I20260811 05:12:49.217741 689 raft_consensus.cc:3037] T 6c6d37788be1427da6d04d4f2675ee7b P d35b5b28f83c408ab44e03ffd27c81e4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:49.217797 916 ts_tablet_manager.cc:1918] T 5c6ed28a34454f57b41b889ba884d304 P 6d7acbb7aab64a218850d78341c533f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 05:12:49.217864 916 log.cc:1196] T 5c6ed28a34454f57b41b889ba884d304 P 6d7acbb7aab64a218850d78341c533f9: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-0/wal/wals/5c6ed28a34454f57b41b889ba884d304
I20260811 05:12:49.218111 916 ts_tablet_manager.cc:1939] T 5c6ed28a34454f57b41b889ba884d304 P 6d7acbb7aab64a218850d78341c533f9: Deleting consensus metadata
I20260811 05:12:49.218286 819 tablet_bootstrap.cc:492] T 6c6d37788be1427da6d04d4f2675ee7b P c80c0a0fe54e4382abe4e7a0345fe9f5: Bootstrap starting.
I20260811 05:12:49.218497 689 raft_consensus.cc:2454] T 6c6d37788be1427da6d04d4f2675ee7b P d35b5b28f83c408ab44e03ffd27c81e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d7acbb7aab64a218850d78341c533f9 in term 1.
I20260811 05:12:49.218665 32534 leader_election.cc:302] T 6c6d37788be1427da6d04d4f2675ee7b P 6d7acbb7aab64a218850d78341c533f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6d7acbb7aab64a218850d78341c533f9, d35b5b28f83c408ab44e03ffd27c81e4; no voters: c80c0a0fe54e4382abe4e7a0345fe9f5
I20260811 05:12:49.218722 819 tablet_bootstrap.cc:654] T 6c6d37788be1427da6d04d4f2675ee7b P c80c0a0fe54e4382abe4e7a0345fe9f5: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:49.219132 840 raft_consensus.cc:2781] T 6c6d37788be1427da6d04d4f2675ee7b P 6d7acbb7aab64a218850d78341c533f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:49.219187 840 raft_consensus.cc:686] T 6c6d37788be1427da6d04d4f2675ee7b P 6d7acbb7aab64a218850d78341c533f9 [term 1 LEADER]: Becoming Leader. State: Replica: 6d7acbb7aab64a218850d78341c533f9, State: Running, Role: LEADER
I20260811 05:12:49.219172 32348 catalog_manager.cc:4841] TS 6d7acbb7aab64a218850d78341c533f9 (127.28.143.129:43067): tablet 5c6ed28a34454f57b41b889ba884d304 (table test-table [id=7fe4524343b04bf8a69a3d13f0e19365]) successfully deleted
I20260811 05:12:49.219265 32348 catalog_manager.cc:4841] TS c80c0a0fe54e4382abe4e7a0345fe9f5 (127.28.143.130:37639): tablet 5c6ed28a34454f57b41b889ba884d304 (table test-table [id=7fe4524343b04bf8a69a3d13f0e19365]) successfully deleted
I20260811 05:12:49.219233 840 consensus_queue.cc:237] T 6c6d37788be1427da6d04d4f2675ee7b P 6d7acbb7aab64a218850d78341c533f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } }
I20260811 05:12:49.219780 32375 catalog_manager.cc:5495] T 6c6d37788be1427da6d04d4f2675ee7b P 6d7acbb7aab64a218850d78341c533f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d7acbb7aab64a218850d78341c533f9 (127.28.143.129). New cstate: current_term: 1 leader_uuid: "6d7acbb7aab64a218850d78341c533f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } health_report { overall_health: UNKNOWN } } }
I20260811 05:12:49.220259 819 tablet_bootstrap.cc:492] T 6c6d37788be1427da6d04d4f2675ee7b P c80c0a0fe54e4382abe4e7a0345fe9f5: No bootstrap required, opened a new log
I20260811 05:12:49.220312 819 ts_tablet_manager.cc:1397] T 6c6d37788be1427da6d04d4f2675ee7b P c80c0a0fe54e4382abe4e7a0345fe9f5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 05:12:49.220464 819 raft_consensus.cc:348] T 6c6d37788be1427da6d04d4f2675ee7b P c80c0a0fe54e4382abe4e7a0345fe9f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } }
I20260811 05:12:49.220527 819 raft_consensus.cc:374] T 6c6d37788be1427da6d04d4f2675ee7b P c80c0a0fe54e4382abe4e7a0345fe9f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:49.220547 819 raft_consensus.cc:729] T 6c6d37788be1427da6d04d4f2675ee7b P c80c0a0fe54e4382abe4e7a0345fe9f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c80c0a0fe54e4382abe4e7a0345fe9f5, State: Initialized, Role: FOLLOWER
I20260811 05:12:49.220590 819 consensus_queue.cc:260] T 6c6d37788be1427da6d04d4f2675ee7b P c80c0a0fe54e4382abe4e7a0345fe9f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } }
I20260811 05:12:49.220856 819 ts_tablet_manager.cc:1428] T 6c6d37788be1427da6d04d4f2675ee7b P c80c0a0fe54e4382abe4e7a0345fe9f5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 05:12:49.229049 449 raft_consensus.cc:3037] T 6c6d37788be1427da6d04d4f2675ee7b P c80c0a0fe54e4382abe4e7a0345fe9f5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:49.229966 449 raft_consensus.cc:1261] T 6c6d37788be1427da6d04d4f2675ee7b P c80c0a0fe54e4382abe4e7a0345fe9f5 [term 1 FOLLOWER]: Refusing update from remote peer 6d7acbb7aab64a218850d78341c533f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:12:49.230234 840 consensus_queue.cc:1035] T 6c6d37788be1427da6d04d4f2675ee7b P 6d7acbb7aab64a218850d78341c533f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:12:49.230372 689 raft_consensus.cc:1261] T 6c6d37788be1427da6d04d4f2675ee7b P d35b5b28f83c408ab44e03ffd27c81e4 [term 1 FOLLOWER]: Refusing update from remote peer 6d7acbb7aab64a218850d78341c533f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:12:49.230639 840 consensus_queue.cc:1035] T 6c6d37788be1427da6d04d4f2675ee7b P 6d7acbb7aab64a218850d78341c533f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 05:12:49.236526 882 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 05:12:50.768893 32375 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:12:50.769011 32375 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:12:50.770000 32375 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 251ba2f17a774486bcc0fa8cfea5c90f: Sending DeleteTablet for 3 replicas of tablet 6c6d37788be1427da6d04d4f2675ee7b
I20260811 05:12:50.770308 32628 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c6d37788be1427da6d04d4f2675ee7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:12:50 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:45924
I20260811 05:12:50.770363 649 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c6d37788be1427da6d04d4f2675ee7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:12:50 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:33680
I20260811 05:12:50.770381 412 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c6d37788be1427da6d04d4f2675ee7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:12:50 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:44154
I20260811 05:12:50.770572 976 tablet_replica.cc:330] T 6c6d37788be1427da6d04d4f2675ee7b P d35b5b28f83c408ab44e03ffd27c81e4: stopping tablet replica
I20260811 05:12:50.770609 977 tablet_replica.cc:330] T 6c6d37788be1427da6d04d4f2675ee7b P c80c0a0fe54e4382abe4e7a0345fe9f5: stopping tablet replica
I20260811 05:12:50.770676 977 raft_consensus.cc:2229] T 6c6d37788be1427da6d04d4f2675ee7b P c80c0a0fe54e4382abe4e7a0345fe9f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:12:50.770721 977 raft_consensus.cc:2258] T 6c6d37788be1427da6d04d4f2675ee7b P c80c0a0fe54e4382abe4e7a0345fe9f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:12:50.770998 977 ts_tablet_manager.cc:1905] T 6c6d37788be1427da6d04d4f2675ee7b P c80c0a0fe54e4382abe4e7a0345fe9f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:12:50.770640 976 raft_consensus.cc:2229] T 6c6d37788be1427da6d04d4f2675ee7b P d35b5b28f83c408ab44e03ffd27c81e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:12:50.771127 976 raft_consensus.cc:2258] T 6c6d37788be1427da6d04d4f2675ee7b P d35b5b28f83c408ab44e03ffd27c81e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:12:50.771384 976 ts_tablet_manager.cc:1905] T 6c6d37788be1427da6d04d4f2675ee7b P d35b5b28f83c408ab44e03ffd27c81e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:12:50.772717 973 tablet_replica.cc:330] T 6c6d37788be1427da6d04d4f2675ee7b P 6d7acbb7aab64a218850d78341c533f9: stopping tablet replica
I20260811 05:12:50.772734 977 ts_tablet_manager.cc:1918] T 6c6d37788be1427da6d04d4f2675ee7b P c80c0a0fe54e4382abe4e7a0345fe9f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 05:12:50.772784 977 log.cc:1196] T 6c6d37788be1427da6d04d4f2675ee7b P c80c0a0fe54e4382abe4e7a0345fe9f5: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-1/wal/wals/6c6d37788be1427da6d04d4f2675ee7b
I20260811 05:12:50.772792 973 raft_consensus.cc:2229] T 6c6d37788be1427da6d04d4f2675ee7b P 6d7acbb7aab64a218850d78341c533f9 [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:12:50.772869 973 raft_consensus.cc:2258] T 6c6d37788be1427da6d04d4f2675ee7b P 6d7acbb7aab64a218850d78341c533f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:12:50.772998 977 ts_tablet_manager.cc:1939] T 6c6d37788be1427da6d04d4f2675ee7b P c80c0a0fe54e4382abe4e7a0345fe9f5: Deleting consensus metadata
I20260811 05:12:50.772979 976 ts_tablet_manager.cc:1918] T 6c6d37788be1427da6d04d4f2675ee7b P d35b5b28f83c408ab44e03ffd27c81e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 05:12:50.773037 976 log.cc:1196] T 6c6d37788be1427da6d04d4f2675ee7b P d35b5b28f83c408ab44e03ffd27c81e4: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-2/wal/wals/6c6d37788be1427da6d04d4f2675ee7b
I20260811 05:12:50.773239 976 ts_tablet_manager.cc:1939] T 6c6d37788be1427da6d04d4f2675ee7b P d35b5b28f83c408ab44e03ffd27c81e4: Deleting consensus metadata
I20260811 05:12:50.773803 32347 catalog_manager.cc:4841] TS d35b5b28f83c408ab44e03ffd27c81e4 (127.28.143.131:38799): tablet 6c6d37788be1427da6d04d4f2675ee7b (table test-table [id=a75a843a4fef4af3b8941889c05d26c5]) successfully deleted
I20260811 05:12:50.774037 973 ts_tablet_manager.cc:1905] T 6c6d37788be1427da6d04d4f2675ee7b P 6d7acbb7aab64a218850d78341c533f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:12:50.774232 32348 catalog_manager.cc:4841] TS c80c0a0fe54e4382abe4e7a0345fe9f5 (127.28.143.130:37639): tablet 6c6d37788be1427da6d04d4f2675ee7b (table test-table [id=a75a843a4fef4af3b8941889c05d26c5]) successfully deleted
I20260811 05:12:50.775553 973 ts_tablet_manager.cc:1918] T 6c6d37788be1427da6d04d4f2675ee7b P 6d7acbb7aab64a218850d78341c533f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 05:12:50.775611 973 log.cc:1196] T 6c6d37788be1427da6d04d4f2675ee7b P 6d7acbb7aab64a218850d78341c533f9: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-0/wal/wals/6c6d37788be1427da6d04d4f2675ee7b
I20260811 05:12:50.775820 973 ts_tablet_manager.cc:1939] T 6c6d37788be1427da6d04d4f2675ee7b P 6d7acbb7aab64a218850d78341c533f9: Deleting consensus metadata
I20260811 05:12:50.776201 32348 catalog_manager.cc:4841] TS 6d7acbb7aab64a218850d78341c533f9 (127.28.143.129:43067): tablet 6c6d37788be1427da6d04d4f2675ee7b (table test-table [id=a75a843a4fef4af3b8941889c05d26c5]) successfully deleted
Aug 11 05:12:50 dist-test-slave-f7th krb5kdc[32291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425170, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 05:12:50 dist-test-slave-f7th krb5kdc[32291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425170, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:12:50.804165 993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:50.790637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43050 (local address 127.28.143.190:44957)
0811 05:12:50.790846 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:50.790849 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:50.790860 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:12:50.790892 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:50.790893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:50.790908 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:50.790992 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:50.791791 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:50.792597 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:50.793677 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:50.793911 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:50.793974 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:50.801162 (+ 7188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:50.801164 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:50.801164 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:50.801175 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:50.802664 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:50.803134 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:50.803136 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:50.803137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:50.803207 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:50.803494 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:50.803496 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:50.803496 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:50.803645 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:50.803742 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:50.803966 (+ 224us) server_negotiation.cc:299] Negotiation successful
0811 05:12:50.804085 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":100,"threads_started":1}
I20260811 05:12:50.806282 32375 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43050:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:12:50.806455 32375 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:12:50.808007 412 tablet_service.cc:1467] Processing CreateTablet for tablet e9df539f6b6d451fbf8f08bc9f19670d (DEFAULT_TABLE table=test-table [id=da2f1569a2c34fa5810d068f0608b225]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:12:50.808005 32628 tablet_service.cc:1467] Processing CreateTablet for tablet e9df539f6b6d451fbf8f08bc9f19670d (DEFAULT_TABLE table=test-table [id=da2f1569a2c34fa5810d068f0608b225]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:12:50.808056 649 tablet_service.cc:1467] Processing CreateTablet for tablet e9df539f6b6d451fbf8f08bc9f19670d (DEFAULT_TABLE table=test-table [id=da2f1569a2c34fa5810d068f0608b225]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:12:50.808173 649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9df539f6b6d451fbf8f08bc9f19670d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:50.808174 32628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9df539f6b6d451fbf8f08bc9f19670d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:50.808174 412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9df539f6b6d451fbf8f08bc9f19670d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:50.809499 994 tablet_bootstrap.cc:492] T e9df539f6b6d451fbf8f08bc9f19670d P c80c0a0fe54e4382abe4e7a0345fe9f5: Bootstrap starting.
I20260811 05:12:50.809760 997 tablet_bootstrap.cc:492] T e9df539f6b6d451fbf8f08bc9f19670d P 6d7acbb7aab64a218850d78341c533f9: Bootstrap starting.
I20260811 05:12:50.809907 994 tablet_bootstrap.cc:654] T e9df539f6b6d451fbf8f08bc9f19670d P c80c0a0fe54e4382abe4e7a0345fe9f5: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:50.810103 997 tablet_bootstrap.cc:654] T e9df539f6b6d451fbf8f08bc9f19670d P 6d7acbb7aab64a218850d78341c533f9: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:50.810417 998 tablet_bootstrap.cc:492] T e9df539f6b6d451fbf8f08bc9f19670d P d35b5b28f83c408ab44e03ffd27c81e4: Bootstrap starting.
I20260811 05:12:50.810636 994 tablet_bootstrap.cc:492] T e9df539f6b6d451fbf8f08bc9f19670d P c80c0a0fe54e4382abe4e7a0345fe9f5: No bootstrap required, opened a new log
I20260811 05:12:50.810696 997 tablet_bootstrap.cc:492] T e9df539f6b6d451fbf8f08bc9f19670d P 6d7acbb7aab64a218850d78341c533f9: No bootstrap required, opened a new log
I20260811 05:12:50.810701 994 ts_tablet_manager.cc:1397] T e9df539f6b6d451fbf8f08bc9f19670d P c80c0a0fe54e4382abe4e7a0345fe9f5: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:12:50.810740 997 ts_tablet_manager.cc:1397] T e9df539f6b6d451fbf8f08bc9f19670d P 6d7acbb7aab64a218850d78341c533f9: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:12:50.810788 998 tablet_bootstrap.cc:654] T e9df539f6b6d451fbf8f08bc9f19670d P d35b5b28f83c408ab44e03ffd27c81e4: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:50.810878 997 raft_consensus.cc:348] T e9df539f6b6d451fbf8f08bc9f19670d P 6d7acbb7aab64a218850d78341c533f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } }
I20260811 05:12:50.810881 994 raft_consensus.cc:348] T e9df539f6b6d451fbf8f08bc9f19670d P c80c0a0fe54e4382abe4e7a0345fe9f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } }
I20260811 05:12:50.810968 994 raft_consensus.cc:374] T e9df539f6b6d451fbf8f08bc9f19670d P c80c0a0fe54e4382abe4e7a0345fe9f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:50.810967 997 raft_consensus.cc:374] T e9df539f6b6d451fbf8f08bc9f19670d P 6d7acbb7aab64a218850d78341c533f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:50.810990 994 raft_consensus.cc:729] T e9df539f6b6d451fbf8f08bc9f19670d P c80c0a0fe54e4382abe4e7a0345fe9f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c80c0a0fe54e4382abe4e7a0345fe9f5, State: Initialized, Role: FOLLOWER
I20260811 05:12:50.810990 997 raft_consensus.cc:729] T e9df539f6b6d451fbf8f08bc9f19670d P 6d7acbb7aab64a218850d78341c533f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d7acbb7aab64a218850d78341c533f9, State: Initialized, Role: FOLLOWER
I20260811 05:12:50.811059 997 consensus_queue.cc:260] T e9df539f6b6d451fbf8f08bc9f19670d P 6d7acbb7aab64a218850d78341c533f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } }
I20260811 05:12:50.811058 994 consensus_queue.cc:260] T e9df539f6b6d451fbf8f08bc9f19670d P c80c0a0fe54e4382abe4e7a0345fe9f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } }
I20260811 05:12:50.811246 994 ts_tablet_manager.cc:1428] T e9df539f6b6d451fbf8f08bc9f19670d P c80c0a0fe54e4382abe4e7a0345fe9f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:12:50.811440 997 ts_tablet_manager.cc:1428] T e9df539f6b6d451fbf8f08bc9f19670d P 6d7acbb7aab64a218850d78341c533f9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:12:50.811710 998 tablet_bootstrap.cc:492] T e9df539f6b6d451fbf8f08bc9f19670d P d35b5b28f83c408ab44e03ffd27c81e4: No bootstrap required, opened a new log
I20260811 05:12:50.811774 998 ts_tablet_manager.cc:1397] T e9df539f6b6d451fbf8f08bc9f19670d P d35b5b28f83c408ab44e03ffd27c81e4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:12:50.812019 998 raft_consensus.cc:348] T e9df539f6b6d451fbf8f08bc9f19670d P d35b5b28f83c408ab44e03ffd27c81e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } }
I20260811 05:12:50.812114 998 raft_consensus.cc:374] T e9df539f6b6d451fbf8f08bc9f19670d P d35b5b28f83c408ab44e03ffd27c81e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:50.812142 998 raft_consensus.cc:729] T e9df539f6b6d451fbf8f08bc9f19670d P d35b5b28f83c408ab44e03ffd27c81e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35b5b28f83c408ab44e03ffd27c81e4, State: Initialized, Role: FOLLOWER
I20260811 05:12:50.812196 998 consensus_queue.cc:260] T e9df539f6b6d451fbf8f08bc9f19670d P d35b5b28f83c408ab44e03ffd27c81e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } }
I20260811 05:12:50.812288 998 ts_tablet_manager.cc:1428] T e9df539f6b6d451fbf8f08bc9f19670d P d35b5b28f83c408ab44e03ffd27c81e4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:50.884755 885 raft_consensus.cc:482] T e9df539f6b6d451fbf8f08bc9f19670d P d35b5b28f83c408ab44e03ffd27c81e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:12:50.884860 885 raft_consensus.cc:504] T e9df539f6b6d451fbf8f08bc9f19670d P d35b5b28f83c408ab44e03ffd27c81e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } }
I20260811 05:12:50.885207 885 leader_election.cc:288] T e9df539f6b6d451fbf8f08bc9f19670d P d35b5b28f83c408ab44e03ffd27c81e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d7acbb7aab64a218850d78341c533f9 (127.28.143.129:43067), c80c0a0fe54e4382abe4e7a0345fe9f5 (127.28.143.130:37639)
I20260811 05:12:50.885536 32654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9df539f6b6d451fbf8f08bc9f19670d" candidate_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d7acbb7aab64a218850d78341c533f9" is_pre_election: true
I20260811 05:12:50.885653 32654 raft_consensus.cc:2454] T e9df539f6b6d451fbf8f08bc9f19670d P 6d7acbb7aab64a218850d78341c533f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35b5b28f83c408ab44e03ffd27c81e4 in term 0.
I20260811 05:12:50.885658 448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9df539f6b6d451fbf8f08bc9f19670d" candidate_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" is_pre_election: true
I20260811 05:12:50.885778 448 raft_consensus.cc:2454] T e9df539f6b6d451fbf8f08bc9f19670d P c80c0a0fe54e4382abe4e7a0345fe9f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d35b5b28f83c408ab44e03ffd27c81e4 in term 0.
I20260811 05:12:50.885960 574 leader_election.cc:302] T e9df539f6b6d451fbf8f08bc9f19670d P d35b5b28f83c408ab44e03ffd27c81e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c80c0a0fe54e4382abe4e7a0345fe9f5, d35b5b28f83c408ab44e03ffd27c81e4; no voters:
I20260811 05:12:50.886091 885 raft_consensus.cc:2781] T e9df539f6b6d451fbf8f08bc9f19670d P d35b5b28f83c408ab44e03ffd27c81e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:12:50.886147 885 raft_consensus.cc:482] T e9df539f6b6d451fbf8f08bc9f19670d P d35b5b28f83c408ab44e03ffd27c81e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:12:50.886173 885 raft_consensus.cc:3037] T e9df539f6b6d451fbf8f08bc9f19670d P d35b5b28f83c408ab44e03ffd27c81e4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:50.886905 885 raft_consensus.cc:504] T e9df539f6b6d451fbf8f08bc9f19670d P d35b5b28f83c408ab44e03ffd27c81e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } }
I20260811 05:12:50.887036 885 leader_election.cc:288] T e9df539f6b6d451fbf8f08bc9f19670d P d35b5b28f83c408ab44e03ffd27c81e4 [CANDIDATE]: Term 1 election: Requested vote from peers 6d7acbb7aab64a218850d78341c533f9 (127.28.143.129:43067), c80c0a0fe54e4382abe4e7a0345fe9f5 (127.28.143.130:37639)
I20260811 05:12:50.887279 448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9df539f6b6d451fbf8f08bc9f19670d" candidate_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5"
I20260811 05:12:50.887265 32654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9df539f6b6d451fbf8f08bc9f19670d" candidate_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d7acbb7aab64a218850d78341c533f9"
I20260811 05:12:50.887367 32654 raft_consensus.cc:3037] T e9df539f6b6d451fbf8f08bc9f19670d P 6d7acbb7aab64a218850d78341c533f9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:50.887368 448 raft_consensus.cc:3037] T e9df539f6b6d451fbf8f08bc9f19670d P c80c0a0fe54e4382abe4e7a0345fe9f5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:50.888307 32654 raft_consensus.cc:2454] T e9df539f6b6d451fbf8f08bc9f19670d P 6d7acbb7aab64a218850d78341c533f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35b5b28f83c408ab44e03ffd27c81e4 in term 1.
I20260811 05:12:50.888324 448 raft_consensus.cc:2454] T e9df539f6b6d451fbf8f08bc9f19670d P c80c0a0fe54e4382abe4e7a0345fe9f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d35b5b28f83c408ab44e03ffd27c81e4 in term 1.
I20260811 05:12:50.888550 574 leader_election.cc:302] T e9df539f6b6d451fbf8f08bc9f19670d P d35b5b28f83c408ab44e03ffd27c81e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c80c0a0fe54e4382abe4e7a0345fe9f5, d35b5b28f83c408ab44e03ffd27c81e4; no voters:
I20260811 05:12:50.888671 885 raft_consensus.cc:2781] T e9df539f6b6d451fbf8f08bc9f19670d P d35b5b28f83c408ab44e03ffd27c81e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:50.888748 885 raft_consensus.cc:686] T e9df539f6b6d451fbf8f08bc9f19670d P d35b5b28f83c408ab44e03ffd27c81e4 [term 1 LEADER]: Becoming Leader. State: Replica: d35b5b28f83c408ab44e03ffd27c81e4, State: Running, Role: LEADER
I20260811 05:12:50.888808 885 consensus_queue.cc:237] T e9df539f6b6d451fbf8f08bc9f19670d P d35b5b28f83c408ab44e03ffd27c81e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } }
I20260811 05:12:50.889581 32375 catalog_manager.cc:5495] T e9df539f6b6d451fbf8f08bc9f19670d P d35b5b28f83c408ab44e03ffd27c81e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to d35b5b28f83c408ab44e03ffd27c81e4 (127.28.143.131). New cstate: current_term: 1 leader_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } health_report { overall_health: HEALTHY } } }
I20260811 05:12:50.911422 1013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:50.907751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47978 (local address 127.28.143.131:38799)
0811 05:12:50.907948 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:50.907950 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:50.907959 (+ 9us) server_negotiation.cc:407] Connection header received
0811 05:12:50.907980 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:50.907981 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:50.907993 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:50.908062 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:12:50.908342 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:50.909074 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:50.910354 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:50.910594 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:50.910660 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:50.910821 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:12:50.910849 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:12:50.910930 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:50.911243 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 05:12:50.911330 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":112,"threads_started":1}
I20260811 05:12:50.917172 1013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:50.914658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:37883 (local address 127.28.143.131:38799)
0811 05:12:50.914735 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:50.914737 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:50.914762 (+ 25us) server_negotiation.cc:407] Connection header received
0811 05:12:50.914824 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:50.914824 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:50.914889 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:50.914944 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:12:50.915086 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:50.915575 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:50.916358 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:50.916547 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:50.916637 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:50.916836 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:12:50.916869 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:12:50.916923 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:50.917027 (+ 104us) server_negotiation.cc:299] Negotiation successful
0811 05:12:50.917083 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 05:12:50.917187 1017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:50.914547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.131:38799 (local address 127.28.143.131:37883)
0811 05:12:50.914708 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:50.914718 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:50.914772 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:50.914952 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:50.914952 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:50.914954 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 05:12:50.914999 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:50.915001 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:50.915656 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:50.915656 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:50.916263 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:50.916265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:50.916670 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:50.916672 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:50.916724 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:50.916730 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 05:12:50.916963 (+ 233us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 05:12:50.916967 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:12:50.917058 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 05:12:50.917107 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":103,"threads_started":1}
I20260811 05:12:50.920353 32654 raft_consensus.cc:1261] T e9df539f6b6d451fbf8f08bc9f19670d P 6d7acbb7aab64a218850d78341c533f9 [term 1 FOLLOWER]: Refusing update from remote peer d35b5b28f83c408ab44e03ffd27c81e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:12:50.920578 448 raft_consensus.cc:1261] T e9df539f6b6d451fbf8f08bc9f19670d P c80c0a0fe54e4382abe4e7a0345fe9f5 [term 1 FOLLOWER]: Refusing update from remote peer d35b5b28f83c408ab44e03ffd27c81e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:12:50.920744 940 consensus_queue.cc:1035] T e9df539f6b6d451fbf8f08bc9f19670d P d35b5b28f83c408ab44e03ffd27c81e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:12:50.920994 940 consensus_queue.cc:1035] T e9df539f6b6d451fbf8f08bc9f19670d P d35b5b28f83c408ab44e03ffd27c81e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:12:50.937688 32375 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:12:50.937830 32375 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:12:50.938613 32375 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 251ba2f17a774486bcc0fa8cfea5c90f: Sending DeleteTablet for 3 replicas of tablet e9df539f6b6d451fbf8f08bc9f19670d
I20260811 05:12:50.938916 32628 tablet_service.cc:1514] Processing DeleteTablet for tablet e9df539f6b6d451fbf8f08bc9f19670d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:12:50 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:45924
I20260811 05:12:50.939019 412 tablet_service.cc:1514] Processing DeleteTablet for tablet e9df539f6b6d451fbf8f08bc9f19670d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:12:50 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:44154
I20260811 05:12:50.939047 649 tablet_service.cc:1514] Processing DeleteTablet for tablet e9df539f6b6d451fbf8f08bc9f19670d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:12:50 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:33680
I20260811 05:12:50.939141 973 tablet_replica.cc:330] T e9df539f6b6d451fbf8f08bc9f19670d P 6d7acbb7aab64a218850d78341c533f9: stopping tablet replica
I20260811 05:12:50.939141 976 tablet_replica.cc:330] T e9df539f6b6d451fbf8f08bc9f19670d P d35b5b28f83c408ab44e03ffd27c81e4: stopping tablet replica
I20260811 05:12:50.939204 976 raft_consensus.cc:2229] T e9df539f6b6d451fbf8f08bc9f19670d P d35b5b28f83c408ab44e03ffd27c81e4 [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:12:50.939203 973 raft_consensus.cc:2229] T e9df539f6b6d451fbf8f08bc9f19670d P 6d7acbb7aab64a218850d78341c533f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:12:50.939254 973 raft_consensus.cc:2258] T e9df539f6b6d451fbf8f08bc9f19670d P 6d7acbb7aab64a218850d78341c533f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:12:50.939332 977 tablet_replica.cc:330] T e9df539f6b6d451fbf8f08bc9f19670d P c80c0a0fe54e4382abe4e7a0345fe9f5: stopping tablet replica
I20260811 05:12:50.939345 976 raft_consensus.cc:2258] T e9df539f6b6d451fbf8f08bc9f19670d P d35b5b28f83c408ab44e03ffd27c81e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:12:50.939396 977 raft_consensus.cc:2229] T e9df539f6b6d451fbf8f08bc9f19670d P c80c0a0fe54e4382abe4e7a0345fe9f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:12:50.939445 977 raft_consensus.cc:2258] T e9df539f6b6d451fbf8f08bc9f19670d P c80c0a0fe54e4382abe4e7a0345fe9f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:12:50.939756 976 ts_tablet_manager.cc:1905] T e9df539f6b6d451fbf8f08bc9f19670d P d35b5b28f83c408ab44e03ffd27c81e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:12:50.939843 977 ts_tablet_manager.cc:1905] T e9df539f6b6d451fbf8f08bc9f19670d P c80c0a0fe54e4382abe4e7a0345fe9f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:12:50.941161 32375 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43050:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:12:50.941311 32375 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:12:50.941692 973 ts_tablet_manager.cc:1905] T e9df539f6b6d451fbf8f08bc9f19670d P 6d7acbb7aab64a218850d78341c533f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:12:50.941701 977 ts_tablet_manager.cc:1918] T e9df539f6b6d451fbf8f08bc9f19670d P c80c0a0fe54e4382abe4e7a0345fe9f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 05:12:50.941763 977 log.cc:1196] T e9df539f6b6d451fbf8f08bc9f19670d P c80c0a0fe54e4382abe4e7a0345fe9f5: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-1/wal/wals/e9df539f6b6d451fbf8f08bc9f19670d
I20260811 05:12:50.941995 977 ts_tablet_manager.cc:1939] T e9df539f6b6d451fbf8f08bc9f19670d P c80c0a0fe54e4382abe4e7a0345fe9f5: Deleting consensus metadata
I20260811 05:12:50.942466 32348 catalog_manager.cc:4841] TS c80c0a0fe54e4382abe4e7a0345fe9f5 (127.28.143.130:37639): tablet e9df539f6b6d451fbf8f08bc9f19670d (table test-table [id=da2f1569a2c34fa5810d068f0608b225]) successfully deleted
I20260811 05:12:50.942739 649 tablet_service.cc:1467] Processing CreateTablet for tablet 3d7b185b141a47babcbf8bb71752f43c (DEFAULT_TABLE table=test-table [id=816f919390d542aabd1dba3a3ceead2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:12:50.942869 412 tablet_service.cc:1467] Processing CreateTablet for tablet 3d7b185b141a47babcbf8bb71752f43c (DEFAULT_TABLE table=test-table [id=816f919390d542aabd1dba3a3ceead2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:12:50.942885 32628 tablet_service.cc:1467] Processing CreateTablet for tablet 3d7b185b141a47babcbf8bb71752f43c (DEFAULT_TABLE table=test-table [id=816f919390d542aabd1dba3a3ceead2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:12:50.942981 32628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d7b185b141a47babcbf8bb71752f43c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:50.943073 412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d7b185b141a47babcbf8bb71752f43c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:50.943327 973 ts_tablet_manager.cc:1918] T e9df539f6b6d451fbf8f08bc9f19670d P 6d7acbb7aab64a218850d78341c533f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 05:12:50.943378 973 log.cc:1196] T e9df539f6b6d451fbf8f08bc9f19670d P 6d7acbb7aab64a218850d78341c533f9: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-0/wal/wals/e9df539f6b6d451fbf8f08bc9f19670d
I20260811 05:12:50.943593 973 ts_tablet_manager.cc:1939] T e9df539f6b6d451fbf8f08bc9f19670d P 6d7acbb7aab64a218850d78341c533f9: Deleting consensus metadata
I20260811 05:12:50.943800 976 ts_tablet_manager.cc:1918] T e9df539f6b6d451fbf8f08bc9f19670d P d35b5b28f83c408ab44e03ffd27c81e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 05:12:50.943852 976 log.cc:1196] T e9df539f6b6d451fbf8f08bc9f19670d P d35b5b28f83c408ab44e03ffd27c81e4: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-2/wal/wals/e9df539f6b6d451fbf8f08bc9f19670d
I20260811 05:12:50.944056 976 ts_tablet_manager.cc:1939] T e9df539f6b6d451fbf8f08bc9f19670d P d35b5b28f83c408ab44e03ffd27c81e4: Deleting consensus metadata
I20260811 05:12:50.944141 997 tablet_bootstrap.cc:492] T 3d7b185b141a47babcbf8bb71752f43c P 6d7acbb7aab64a218850d78341c533f9: Bootstrap starting.
I20260811 05:12:50.944424 994 tablet_bootstrap.cc:492] T 3d7b185b141a47babcbf8bb71752f43c P c80c0a0fe54e4382abe4e7a0345fe9f5: Bootstrap starting.
I20260811 05:12:50.944507 997 tablet_bootstrap.cc:654] T 3d7b185b141a47babcbf8bb71752f43c P 6d7acbb7aab64a218850d78341c533f9: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:50.944901 994 tablet_bootstrap.cc:654] T 3d7b185b141a47babcbf8bb71752f43c P c80c0a0fe54e4382abe4e7a0345fe9f5: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:50.944978 649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d7b185b141a47babcbf8bb71752f43c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:50.945004 32348 catalog_manager.cc:4841] TS 6d7acbb7aab64a218850d78341c533f9 (127.28.143.129:43067): tablet e9df539f6b6d451fbf8f08bc9f19670d (table test-table [id=da2f1569a2c34fa5810d068f0608b225]) successfully deleted
I20260811 05:12:50.945458 32347 catalog_manager.cc:4841] TS d35b5b28f83c408ab44e03ffd27c81e4 (127.28.143.131:38799): tablet e9df539f6b6d451fbf8f08bc9f19670d (table test-table [id=da2f1569a2c34fa5810d068f0608b225]) successfully deleted
I20260811 05:12:50.945549 994 tablet_bootstrap.cc:492] T 3d7b185b141a47babcbf8bb71752f43c P c80c0a0fe54e4382abe4e7a0345fe9f5: No bootstrap required, opened a new log
I20260811 05:12:50.945596 994 ts_tablet_manager.cc:1397] T 3d7b185b141a47babcbf8bb71752f43c P c80c0a0fe54e4382abe4e7a0345fe9f5: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260811 05:12:50.945757 994 raft_consensus.cc:348] T 3d7b185b141a47babcbf8bb71752f43c P c80c0a0fe54e4382abe4e7a0345fe9f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } }
I20260811 05:12:50.945827 994 raft_consensus.cc:374] T 3d7b185b141a47babcbf8bb71752f43c P c80c0a0fe54e4382abe4e7a0345fe9f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:50.945844 994 raft_consensus.cc:729] T 3d7b185b141a47babcbf8bb71752f43c P c80c0a0fe54e4382abe4e7a0345fe9f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c80c0a0fe54e4382abe4e7a0345fe9f5, State: Initialized, Role: FOLLOWER
I20260811 05:12:50.945896 994 consensus_queue.cc:260] T 3d7b185b141a47babcbf8bb71752f43c P c80c0a0fe54e4382abe4e7a0345fe9f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } }
I20260811 05:12:50.945990 994 ts_tablet_manager.cc:1428] T 3d7b185b141a47babcbf8bb71752f43c P c80c0a0fe54e4382abe4e7a0345fe9f5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:50.946171 998 tablet_bootstrap.cc:492] T 3d7b185b141a47babcbf8bb71752f43c P d35b5b28f83c408ab44e03ffd27c81e4: Bootstrap starting.
I20260811 05:12:50.946471 998 tablet_bootstrap.cc:654] T 3d7b185b141a47babcbf8bb71752f43c P d35b5b28f83c408ab44e03ffd27c81e4: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:50.946686 997 tablet_bootstrap.cc:492] T 3d7b185b141a47babcbf8bb71752f43c P 6d7acbb7aab64a218850d78341c533f9: No bootstrap required, opened a new log
I20260811 05:12:50.946754 997 ts_tablet_manager.cc:1397] T 3d7b185b141a47babcbf8bb71752f43c P 6d7acbb7aab64a218850d78341c533f9: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260811 05:12:50.946906 997 raft_consensus.cc:348] T 3d7b185b141a47babcbf8bb71752f43c P 6d7acbb7aab64a218850d78341c533f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } }
I20260811 05:12:50.946997 997 raft_consensus.cc:374] T 3d7b185b141a47babcbf8bb71752f43c P 6d7acbb7aab64a218850d78341c533f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:50.947022 997 raft_consensus.cc:729] T 3d7b185b141a47babcbf8bb71752f43c P 6d7acbb7aab64a218850d78341c533f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d7acbb7aab64a218850d78341c533f9, State: Initialized, Role: FOLLOWER
I20260811 05:12:50.947083 997 consensus_queue.cc:260] T 3d7b185b141a47babcbf8bb71752f43c P 6d7acbb7aab64a218850d78341c533f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } }
I20260811 05:12:50.947134 998 tablet_bootstrap.cc:492] T 3d7b185b141a47babcbf8bb71752f43c P d35b5b28f83c408ab44e03ffd27c81e4: No bootstrap required, opened a new log
I20260811 05:12:50.947185 997 ts_tablet_manager.cc:1428] T 3d7b185b141a47babcbf8bb71752f43c P 6d7acbb7aab64a218850d78341c533f9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:50.947193 998 ts_tablet_manager.cc:1397] T 3d7b185b141a47babcbf8bb71752f43c P d35b5b28f83c408ab44e03ffd27c81e4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:12:50.947320 998 raft_consensus.cc:348] T 3d7b185b141a47babcbf8bb71752f43c P d35b5b28f83c408ab44e03ffd27c81e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } }
I20260811 05:12:50.947393 998 raft_consensus.cc:374] T 3d7b185b141a47babcbf8bb71752f43c P d35b5b28f83c408ab44e03ffd27c81e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:50.947412 998 raft_consensus.cc:729] T 3d7b185b141a47babcbf8bb71752f43c P d35b5b28f83c408ab44e03ffd27c81e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35b5b28f83c408ab44e03ffd27c81e4, State: Initialized, Role: FOLLOWER
I20260811 05:12:50.947450 998 consensus_queue.cc:260] T 3d7b185b141a47babcbf8bb71752f43c P d35b5b28f83c408ab44e03ffd27c81e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } }
I20260811 05:12:50.947542 998 ts_tablet_manager.cc:1428] T 3d7b185b141a47babcbf8bb71752f43c P d35b5b28f83c408ab44e03ffd27c81e4: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 05:12:51.154783 1006 raft_consensus.cc:482] T 3d7b185b141a47babcbf8bb71752f43c P c80c0a0fe54e4382abe4e7a0345fe9f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:12:51.154953 1006 raft_consensus.cc:504] T 3d7b185b141a47babcbf8bb71752f43c P c80c0a0fe54e4382abe4e7a0345fe9f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } }
I20260811 05:12:51.155346 1006 leader_election.cc:288] T 3d7b185b141a47babcbf8bb71752f43c P c80c0a0fe54e4382abe4e7a0345fe9f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d7acbb7aab64a218850d78341c533f9 (127.28.143.129:43067), d35b5b28f83c408ab44e03ffd27c81e4 (127.28.143.131:38799)
I20260811 05:12:51.159660 1031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:51.155708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:58033 (local address 127.28.143.129:43067)
0811 05:12:51.155823 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:51.155824 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:51.155833 (+ 9us) server_negotiation.cc:407] Connection header received
0811 05:12:51.155893 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:51.155894 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:51.155905 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:51.155996 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:51.156197 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:51.157011 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:51.158579 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:51.159196 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:51.159290 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:51.159300 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:51.159508 (+ 208us) server_negotiation.cc:299] Negotiation successful
0811 05:12:51.159564 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":62,"threads_started":1}
I20260811 05:12:51.159646 1013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:51.155704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:53631 (local address 127.28.143.131:38799)
0811 05:12:51.155756 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:51.155758 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:51.155832 (+ 74us) server_negotiation.cc:407] Connection header received
0811 05:12:51.155895 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:51.155896 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:51.155904 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:51.156010 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:12:51.156188 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:51.157017 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:51.158579 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:51.159205 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:51.159289 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:51.159300 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:51.159508 (+ 208us) server_negotiation.cc:299] Negotiation successful
0811 05:12:51.159563 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 05:12:51.159711 1028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:51.155600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.131:38799 (local address 127.28.143.130:53631)
0811 05:12:51.155763 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:51.155774 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:51.155833 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:51.156017 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:51.156017 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:51.156025 (+ 8us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:51.156116 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:51.156117 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:51.157148 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:51.157149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:51.158450 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:51.158454 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:51.159296 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:51.159297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:51.159399 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:51.159407 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 05:12:51.159466 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 05:12:51.159569 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":52,"thread_start_us":94,"threads_started":1}
I20260811 05:12:51.159984 689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d7b185b141a47babcbf8bb71752f43c" candidate_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" is_pre_election: true
I20260811 05:12:51.159714 1029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:51.155600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:43067 (local address 127.28.143.130:58033)
0811 05:12:51.155763 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:51.155774 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:51.155832 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:51.156016 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:51.156017 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:51.156019 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:12:51.156104 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:51.156107 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:51.157127 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:51.157128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:51.158450 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:51.158454 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:51.159295 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:51.159296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:51.159399 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:51.159406 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 05:12:51.159466 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 05:12:51.159568 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":44,"thread_start_us":71,"threads_started":1}
I20260811 05:12:51.160076 689 raft_consensus.cc:2454] T 3d7b185b141a47babcbf8bb71752f43c P d35b5b28f83c408ab44e03ffd27c81e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c80c0a0fe54e4382abe4e7a0345fe9f5 in term 0.
I20260811 05:12:51.160272 32654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d7b185b141a47babcbf8bb71752f43c" candidate_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d7acbb7aab64a218850d78341c533f9" is_pre_election: true
I20260811 05:12:51.160297 322 leader_election.cc:302] T 3d7b185b141a47babcbf8bb71752f43c P c80c0a0fe54e4382abe4e7a0345fe9f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c80c0a0fe54e4382abe4e7a0345fe9f5, d35b5b28f83c408ab44e03ffd27c81e4; no voters:
I20260811 05:12:51.160379 32654 raft_consensus.cc:2454] T 3d7b185b141a47babcbf8bb71752f43c P 6d7acbb7aab64a218850d78341c533f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c80c0a0fe54e4382abe4e7a0345fe9f5 in term 0.
I20260811 05:12:51.160476 1006 raft_consensus.cc:2781] T 3d7b185b141a47babcbf8bb71752f43c P c80c0a0fe54e4382abe4e7a0345fe9f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:12:51.160535 1006 raft_consensus.cc:482] T 3d7b185b141a47babcbf8bb71752f43c P c80c0a0fe54e4382abe4e7a0345fe9f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:12:51.160557 1006 raft_consensus.cc:3037] T 3d7b185b141a47babcbf8bb71752f43c P c80c0a0fe54e4382abe4e7a0345fe9f5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:51.161338 1006 raft_consensus.cc:504] T 3d7b185b141a47babcbf8bb71752f43c P c80c0a0fe54e4382abe4e7a0345fe9f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } }
I20260811 05:12:51.161509 1006 leader_election.cc:288] T 3d7b185b141a47babcbf8bb71752f43c P c80c0a0fe54e4382abe4e7a0345fe9f5 [CANDIDATE]: Term 1 election: Requested vote from peers 6d7acbb7aab64a218850d78341c533f9 (127.28.143.129:43067), d35b5b28f83c408ab44e03ffd27c81e4 (127.28.143.131:38799)
I20260811 05:12:51.161697 32654 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d7b185b141a47babcbf8bb71752f43c" candidate_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d7acbb7aab64a218850d78341c533f9"
I20260811 05:12:51.161741 689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d7b185b141a47babcbf8bb71752f43c" candidate_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35b5b28f83c408ab44e03ffd27c81e4"
I20260811 05:12:51.161790 32654 raft_consensus.cc:3037] T 3d7b185b141a47babcbf8bb71752f43c P 6d7acbb7aab64a218850d78341c533f9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:51.161804 689 raft_consensus.cc:3037] T 3d7b185b141a47babcbf8bb71752f43c P d35b5b28f83c408ab44e03ffd27c81e4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:51.162416 689 raft_consensus.cc:2454] T 3d7b185b141a47babcbf8bb71752f43c P d35b5b28f83c408ab44e03ffd27c81e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c80c0a0fe54e4382abe4e7a0345fe9f5 in term 1.
I20260811 05:12:51.162606 322 leader_election.cc:302] T 3d7b185b141a47babcbf8bb71752f43c P c80c0a0fe54e4382abe4e7a0345fe9f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c80c0a0fe54e4382abe4e7a0345fe9f5, d35b5b28f83c408ab44e03ffd27c81e4; no voters:
I20260811 05:12:51.162726 1006 raft_consensus.cc:2781] T 3d7b185b141a47babcbf8bb71752f43c P c80c0a0fe54e4382abe4e7a0345fe9f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:51.162775 32654 raft_consensus.cc:2454] T 3d7b185b141a47babcbf8bb71752f43c P 6d7acbb7aab64a218850d78341c533f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c80c0a0fe54e4382abe4e7a0345fe9f5 in term 1.
I20260811 05:12:51.162910 1006 raft_consensus.cc:686] T 3d7b185b141a47babcbf8bb71752f43c P c80c0a0fe54e4382abe4e7a0345fe9f5 [term 1 LEADER]: Becoming Leader. State: Replica: c80c0a0fe54e4382abe4e7a0345fe9f5, State: Running, Role: LEADER
I20260811 05:12:51.163060 1006 consensus_queue.cc:237] T 3d7b185b141a47babcbf8bb71752f43c P c80c0a0fe54e4382abe4e7a0345fe9f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } }
I20260811 05:12:51.163964 32377 catalog_manager.cc:5495] T 3d7b185b141a47babcbf8bb71752f43c P c80c0a0fe54e4382abe4e7a0345fe9f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to c80c0a0fe54e4382abe4e7a0345fe9f5 (127.28.143.130). New cstate: current_term: 1 leader_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c80c0a0fe54e4382abe4e7a0345fe9f5" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 } health_report { overall_health: UNKNOWN } } }
I20260811 05:12:51.183252 1037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:51.180703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40548 (local address 127.28.143.130:37639)
0811 05:12:51.180875 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:51.180877 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:51.180882 (+ 5us) server_negotiation.cc:407] Connection header received
0811 05:12:51.180918 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:51.180918 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:51.180937 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:51.181001 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:12:51.181126 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:51.181621 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:51.182427 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:51.182652 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:51.182713 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:51.182860 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:12:51.182975 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:12:51.183075 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:51.183149 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 05:12:51.183170 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":119,"threads_started":1}
I20260811 05:12:51.188438 1013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:51.186221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:58325 (local address 127.28.143.131:38799)
0811 05:12:51.186293 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:51.186295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:51.186303 (+ 8us) server_negotiation.cc:407] Connection header received
0811 05:12:51.186326 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:51.186326 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:51.186338 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:51.186380 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:12:51.186531 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:51.187007 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:51.187763 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:51.187953 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:51.188008 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:51.188202 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:12:51.188231 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:12:51.188273 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:51.188330 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 05:12:51.188353 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 05:12:51.188452 1043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:51.186025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.131:38799 (local address 127.28.143.130:58325)
0811 05:12:51.186191 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:51.186202 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:51.186248 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:51.186390 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:51.186391 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:51.186392 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 05:12:51.186462 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:51.186464 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:51.187079 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:51.187080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:51.187676 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:51.187679 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:51.188032 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:51.188033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:51.188106 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:51.188123 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 05:12:51.188280 (+ 157us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 05:12:51.188282 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 05:12:51.188318 (+ 36us) client_negotiation.cc:240] Negotiation successful
0811 05:12:51.188365 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":79,"threads_started":1}
I20260811 05:12:51.190686 689 raft_consensus.cc:1261] T 3d7b185b141a47babcbf8bb71752f43c P d35b5b28f83c408ab44e03ffd27c81e4 [term 1 FOLLOWER]: Refusing update from remote peer c80c0a0fe54e4382abe4e7a0345fe9f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:12:51.190791 32654 raft_consensus.cc:1261] T 3d7b185b141a47babcbf8bb71752f43c P 6d7acbb7aab64a218850d78341c533f9 [term 1 FOLLOWER]: Refusing update from remote peer c80c0a0fe54e4382abe4e7a0345fe9f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:12:51.191264 1034 consensus_queue.cc:1035] T 3d7b185b141a47babcbf8bb71752f43c P c80c0a0fe54e4382abe4e7a0345fe9f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35b5b28f83c408ab44e03ffd27c81e4" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 38799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:12:51.191462 1006 consensus_queue.cc:1035] T 3d7b185b141a47babcbf8bb71752f43c P c80c0a0fe54e4382abe4e7a0345fe9f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d7acbb7aab64a218850d78341c533f9" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:12:51.198912 1017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:51.196556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:37639 (local address 127.28.143.131:39287)
0811 05:12:51.196630 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:51.196643 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:51.196693 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:51.196811 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:51.196812 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:51.196813 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 05:12:51.196859 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:51.196861 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:51.197540 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:51.197541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:51.198088 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:51.198091 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:51.198463 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:51.198463 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:51.198576 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:51.198586 (+ 10us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 05:12:51.198739 (+ 153us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 05:12:51.198742 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 05:12:51.198781 (+ 39us) client_negotiation.cc:240] Negotiation successful
0811 05:12:51.198835 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260811 05:12:51.198912 1037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:51.196598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:39287 (local address 127.28.143.130:37639)
0811 05:12:51.196652 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:51.196654 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:51.196675 (+ 21us) server_negotiation.cc:407] Connection header received
0811 05:12:51.196741 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:51.196742 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:51.196752 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:51.196800 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:12:51.196926 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:51.197467 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:51.198171 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:51.198371 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:51.198430 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:51.198651 (+ 221us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:12:51.198687 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:12:51.198732 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:51.198802 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 05:12:51.198824 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260811 05:12:51.199290 941 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 05:12:52.721267 32375 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:12:52.721380 32375 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:12:52.722291 32375 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 251ba2f17a774486bcc0fa8cfea5c90f: Sending DeleteTablet for 3 replicas of tablet 3d7b185b141a47babcbf8bb71752f43c
I20260811 05:12:52.722585 649 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d7b185b141a47babcbf8bb71752f43c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:12:52 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:33680
I20260811 05:12:52.722669 412 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d7b185b141a47babcbf8bb71752f43c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:12:52 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:44154
I20260811 05:12:52.722657 32628 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d7b185b141a47babcbf8bb71752f43c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:12:52 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:45924
I20260811 05:12:52.722707 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 32501
I20260811 05:12:52.731916 1089 tablet_replica.cc:330] T 3d7b185b141a47babcbf8bb71752f43c P c80c0a0fe54e4382abe4e7a0345fe9f5: stopping tablet replica
I20260811 05:12:52.732029 1089 raft_consensus.cc:2229] T 3d7b185b141a47babcbf8bb71752f43c P c80c0a0fe54e4382abe4e7a0345fe9f5 [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:12:52.732141 1089 raft_consensus.cc:2258] T 3d7b185b141a47babcbf8bb71752f43c P c80c0a0fe54e4382abe4e7a0345fe9f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:12:52.732276 1087 tablet_replica.cc:330] T 3d7b185b141a47babcbf8bb71752f43c P d35b5b28f83c408ab44e03ffd27c81e4: stopping tablet replica
I20260811 05:12:52.732332 1087 raft_consensus.cc:2229] T 3d7b185b141a47babcbf8bb71752f43c P d35b5b28f83c408ab44e03ffd27c81e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:12:52.732372 1087 raft_consensus.cc:2258] T 3d7b185b141a47babcbf8bb71752f43c P d35b5b28f83c408ab44e03ffd27c81e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:12:52.732641 1089 ts_tablet_manager.cc:1905] T 3d7b185b141a47babcbf8bb71752f43c P c80c0a0fe54e4382abe4e7a0345fe9f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:12:52.734160 1087 ts_tablet_manager.cc:1905] T 3d7b185b141a47babcbf8bb71752f43c P d35b5b28f83c408ab44e03ffd27c81e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:12:52.734481 1089 ts_tablet_manager.cc:1918] T 3d7b185b141a47babcbf8bb71752f43c P c80c0a0fe54e4382abe4e7a0345fe9f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 05:12:52.734550 1089 log.cc:1196] T 3d7b185b141a47babcbf8bb71752f43c P c80c0a0fe54e4382abe4e7a0345fe9f5: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-1/wal/wals/3d7b185b141a47babcbf8bb71752f43c
I20260811 05:12:52.734771 1089 ts_tablet_manager.cc:1939] T 3d7b185b141a47babcbf8bb71752f43c P c80c0a0fe54e4382abe4e7a0345fe9f5: Deleting consensus metadata
I20260811 05:12:52.735733 1087 ts_tablet_manager.cc:1918] T 3d7b185b141a47babcbf8bb71752f43c P d35b5b28f83c408ab44e03ffd27c81e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 05:12:52.735808 1087 log.cc:1196] T 3d7b185b141a47babcbf8bb71752f43c P d35b5b28f83c408ab44e03ffd27c81e4: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425159060450-29246-0/minicluster-data/ts-2/wal/wals/3d7b185b141a47babcbf8bb71752f43c
I20260811 05:12:52.736029 1087 ts_tablet_manager.cc:1939] T 3d7b185b141a47babcbf8bb71752f43c P d35b5b28f83c408ab44e03ffd27c81e4: Deleting consensus metadata
I20260811 05:12:52.736449 32347 catalog_manager.cc:4841] TS d35b5b28f83c408ab44e03ffd27c81e4 (127.28.143.131:38799): tablet 3d7b185b141a47babcbf8bb71752f43c (table test-table [id=816f919390d542aabd1dba3a3ceead2d]) successfully deleted
I20260811 05:12:52.736754 32348 catalog_manager.cc:4841] TS c80c0a0fe54e4382abe4e7a0345fe9f5 (127.28.143.130:37639): tablet 3d7b185b141a47babcbf8bb71752f43c (table test-table [id=816f919390d542aabd1dba3a3ceead2d]) successfully deleted
W20260811 05:12:52.737808 32348 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.143.129:43067 (error 108)
I20260811 05:12:52.738087 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 32757
I20260811 05:12:52.741552 1093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:52.738205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:43067 (local address 127.0.0.1:35614)
0811 05:12:52.741440 (+ 3235us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:52.741506 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.129:43067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3200,"thread_start_us":63,"threads_started":1}
W20260811 05:12:52.741703 32348 catalog_manager.cc:4584] TS 6d7acbb7aab64a218850d78341c533f9 (127.28.143.129:43067): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3d7b185b141a47babcbf8bb71752f43c: Network error: Client connection negotiation failed: client connection to 127.28.143.129:43067: connect: Connection refused (error 111)
I20260811 05:12:52.753252 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 541
I20260811 05:12:52.776597 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 32313
2026-08-11T05:12:52Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5969 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:12:52 dist-test-slave-f7th krb5kdc[1096](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:12:52 dist-test-slave-f7th krb5kdc[1096](info): set up 2 sockets
Aug 11 05:12:52 dist-test-slave-f7th krb5kdc[1096](info): commencing operation
krb5kdc: starting...
W20260811 05:12:54.011004 29246 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Aug 11 05:12:54 dist-test-slave-f7th krb5kdc[1096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425174, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:12:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:12:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
I20260811 05:12:54.151499 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:39113
--webserver_interface=127.28.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:33247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:39113
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:54.238679 1115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:54.238931 1115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:54.238966 1115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:54.240245 1115 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:12:54.240302 1115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:54.240324 1115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:54.240339 1115 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:12:54.240355 1115 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:12:54.242153 1115 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:33247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:39113
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:39113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:12:54.242466 1115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:54.242722 1115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:54.248306 1132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:54.248306 1126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:54.248332 1129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:54.248389 1115 server_base.cc:1034] running on GCE node
I20260811 05:12:54.248808 1115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:54.249163 1115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:54.250342 1115 hybrid_clock.cc:648] HybridClock initialized: now 1786425174250313 us; error 45 us; skew 500 ppm
Aug 11 05:12:54 dist-test-slave-f7th krb5kdc[1096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425174, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:54.253219 1115 init.cc:377] Logged in from keytab as kudu/127.28.143.190@KRBTEST.COM (short username kudu)
I20260811 05:12:54.253690 1115 webserver.cc:460] Webserver started at http://127.28.143.190:32947/ using document root <none> and password file <none>
I20260811 05:12:54.253943 1115 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:54.253994 1115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:54.254110 1115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:54.255213 1115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "c9f9d9f4a79c4893b8ebb5428399848d"
format_stamp: "Formatted at 2026-08-11 05:12:54 on dist-test-slave-f7th"
server_key: "1cdc6827f2f516e3343235134833e6d4"
server_key_iv: "9bd27ceff3757d810fce1c99a9dfd43f"
server_key_version: "encryptionkey@0"
I20260811 05:12:54.255621 1115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "c9f9d9f4a79c4893b8ebb5428399848d"
format_stamp: "Formatted at 2026-08-11 05:12:54 on dist-test-slave-f7th"
server_key: "1cdc6827f2f516e3343235134833e6d4"
server_key_iv: "9bd27ceff3757d810fce1c99a9dfd43f"
server_key_version: "encryptionkey@0"
I20260811 05:12:54.257153 1115 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 05:12:54.258087 1142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:54.258400 1115 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 05:12:54.258512 1115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "c9f9d9f4a79c4893b8ebb5428399848d"
format_stamp: "Formatted at 2026-08-11 05:12:54 on dist-test-slave-f7th"
server_key: "1cdc6827f2f516e3343235134833e6d4"
server_key_iv: "9bd27ceff3757d810fce1c99a9dfd43f"
server_key_version: "encryptionkey@0"
I20260811 05:12:54.258585 1115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:54.277669 1115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:54.279731 1115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:54.279886 1115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:54.284397 1115 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:39113
I20260811 05:12:54.284461 1227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:39113 every 8 connection(s)
I20260811 05:12:54.284843 1115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:12:54.285933 1228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:54.288034 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 1115
I20260811 05:12:54.288184 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:12:54.288335 29246 external_mini_cluster.cc:1428] Setting key 36f6420dd8df3cc91e181f396219ccfe
I20260811 05:12:54.288849 1228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9f9d9f4a79c4893b8ebb5428399848d: Bootstrap starting.
I20260811 05:12:54.289577 1228 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9f9d9f4a79c4893b8ebb5428399848d: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:54.289884 1228 log.cc:824] T 00000000000000000000000000000000 P c9f9d9f4a79c4893b8ebb5428399848d: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:54.290642 1228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9f9d9f4a79c4893b8ebb5428399848d: No bootstrap required, opened a new log
I20260811 05:12:54.292579 1228 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9f9d9f4a79c4893b8ebb5428399848d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9f9d9f4a79c4893b8ebb5428399848d" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39113 } }
I20260811 05:12:54.292788 1228 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9f9d9f4a79c4893b8ebb5428399848d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:54.292837 1228 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9f9d9f4a79c4893b8ebb5428399848d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9f9d9f4a79c4893b8ebb5428399848d, State: Initialized, Role: FOLLOWER
I20260811 05:12:54.292977 1228 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9f9d9f4a79c4893b8ebb5428399848d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9f9d9f4a79c4893b8ebb5428399848d" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39113 } }
I20260811 05:12:54.293056 1228 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9f9d9f4a79c4893b8ebb5428399848d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:12:54.293118 1228 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9f9d9f4a79c4893b8ebb5428399848d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:12:54.293174 1228 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9f9d9f4a79c4893b8ebb5428399848d [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:54.293933 1228 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9f9d9f4a79c4893b8ebb5428399848d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9f9d9f4a79c4893b8ebb5428399848d" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39113 } }
I20260811 05:12:54.294046 1228 leader_election.cc:302] T 00000000000000000000000000000000 P c9f9d9f4a79c4893b8ebb5428399848d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9f9d9f4a79c4893b8ebb5428399848d; no voters:
I20260811 05:12:54.294250 1228 leader_election.cc:288] T 00000000000000000000000000000000 P c9f9d9f4a79c4893b8ebb5428399848d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:12:54.294293 1240 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9f9d9f4a79c4893b8ebb5428399848d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:54.294410 1240 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9f9d9f4a79c4893b8ebb5428399848d [term 1 LEADER]: Becoming Leader. State: Replica: c9f9d9f4a79c4893b8ebb5428399848d, State: Running, Role: LEADER
I20260811 05:12:54.294576 1228 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9f9d9f4a79c4893b8ebb5428399848d [sys.catalog]: configured and running, proceeding with master startup.
Aug 11 05:12:54 dist-test-slave-f7th krb5kdc[1096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425174, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:12:54.294569 1240 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9f9d9f4a79c4893b8ebb5428399848d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9f9d9f4a79c4893b8ebb5428399848d" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39113 } }
I20260811 05:12:54.295979 1241 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9f9d9f4a79c4893b8ebb5428399848d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9f9d9f4a79c4893b8ebb5428399848d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9f9d9f4a79c4893b8ebb5428399848d" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39113 } } }
I20260811 05:12:54.296094 1241 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9f9d9f4a79c4893b8ebb5428399848d [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:54.296324 1243 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9f9d9f4a79c4893b8ebb5428399848d [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9f9d9f4a79c4893b8ebb5428399848d. Latest consensus state: current_term: 1 leader_uuid: "c9f9d9f4a79c4893b8ebb5428399848d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9f9d9f4a79c4893b8ebb5428399848d" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39113 } } }
I20260811 05:12:54.296387 1243 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9f9d9f4a79c4893b8ebb5428399848d [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:54.296896 1262 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:12:54.297487 1262 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:12:54.299978 1232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:54.289040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53092 (local address 127.28.143.190:39113)
0811 05:12:54.289248 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:54.289257 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:54.289285 (+ 28us) server_negotiation.cc:407] Connection header received
0811 05:12:54.289822 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:54.289831 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:54.289886 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:54.290025 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:54.290714 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:54.291659 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:54.292431 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:54.292963 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:54.293070 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:54.295763 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:54.295769 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:54.295769 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:54.295793 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:54.298093 (+ 2300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:54.298496 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:54.298500 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:54.298501 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:54.298598 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:54.298791 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:54.298792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:54.298792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:54.299111 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:54.299272 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:54.299582 (+ 310us) server_negotiation.cc:299] Negotiation successful
0811 05:12:54.299751 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":94,"threads_started":1}
I20260811 05:12:54.300130 1262 catalog_manager.cc:1347] Generated new cluster ID: c2565a9389f4484780a9ea683033160e
I20260811 05:12:54.300166 1262 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:12:54.310285 1262 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:12:54.310886 1262 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:12:54.321863 1262 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9f9d9f4a79c4893b8ebb5428399848d: Generated new TSK 0
I20260811 05:12:54.322085 1262 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.143.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab.
Entry for principal kudu/127.28.143.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab.
I20260811 05:12:54.355252 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.129:0
--local_ip_for_outbound_sockets=127.28.143.129
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:39113
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:33247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:54.471220 1283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:54.471488 1283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:54.471522 1283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:54.472944 1283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:54.473008 1283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:54.473042 1283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.129
I20260811 05:12:54.474778 1283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:33247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:39113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.129
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:12:54.475207 1283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:54.475564 1283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:54.482189 1289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:54.482204 1288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:54.482192 1291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:54.482577 1283 server_base.cc:1034] running on GCE node
I20260811 05:12:54.482861 1283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:54.483151 1283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:54.484357 1283 hybrid_clock.cc:648] HybridClock initialized: now 1786425174484244 us; error 125 us; skew 500 ppm
Aug 11 05:12:54 dist-test-slave-f7th krb5kdc[1096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425174, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:54.488562 1283 init.cc:377] Logged in from keytab as kudu/127.28.143.129@KRBTEST.COM (short username kudu)
I20260811 05:12:54.489192 1283 webserver.cc:460] Webserver started at http://127.28.143.129:33175/ using document root <none> and password file <none>
I20260811 05:12:54.489439 1283 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:54.489509 1283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:54.489682 1283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:54.490829 1283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-0/data/instance:
uuid: "530eb154eba04f84a1be17f6fcff3947"
format_stamp: "Formatted at 2026-08-11 05:12:54 on dist-test-slave-f7th"
server_key: "382e74f726e9c0fa1211689345df3ff6"
server_key_iv: "d5b8e7cdee0d6ba454f4715b440b67b1"
server_key_version: "encryptionkey@0"
I20260811 05:12:54.491207 1283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance:
uuid: "530eb154eba04f84a1be17f6fcff3947"
format_stamp: "Formatted at 2026-08-11 05:12:54 on dist-test-slave-f7th"
server_key: "382e74f726e9c0fa1211689345df3ff6"
server_key_iv: "d5b8e7cdee0d6ba454f4715b440b67b1"
server_key_version: "encryptionkey@0"
I20260811 05:12:54.492735 1283 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 05:12:54.493709 1309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:54.493924 1283 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:12:54.494025 1283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-0/wal
uuid: "530eb154eba04f84a1be17f6fcff3947"
format_stamp: "Formatted at 2026-08-11 05:12:54 on dist-test-slave-f7th"
server_key: "382e74f726e9c0fa1211689345df3ff6"
server_key_iv: "d5b8e7cdee0d6ba454f4715b440b67b1"
server_key_version: "encryptionkey@0"
I20260811 05:12:54.494112 1283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:54.510965 1283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:54.513777 1283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:54.514118 1283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:54.514621 1283 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:12:54.515139 1283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:12:54.515187 1283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:54.515223 1283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:12:54.515239 1283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:54.521725 1283 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.129:32985
I20260811 05:12:54.521958 1519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.129:32985 every 8 connection(s)
I20260811 05:12:54.522220 1283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
I20260811 05:12:54.522966 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 1283
I20260811 05:12:54.523054 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance
I20260811 05:12:54.523254 29246 external_mini_cluster.cc:1428] Setting key 12045edd0cc3ead0383b42b96ff515dc
Aug 11 05:12:54 dist-test-slave-f7th krb5kdc[1096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425174, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.28.143.130@KRBTEST.COM; defaulting to no policy
I20260811 05:12:54.534225 1232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:54.523151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:55461 (local address 127.28.143.190:39113)
0811 05:12:54.523223 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:54.523226 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:54.523615 (+ 389us) server_negotiation.cc:407] Connection header received
0811 05:12:54.524217 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:54.524218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:54.524232 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:54.524305 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:54.525720 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:54.526317 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:54.527816 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:54.528070 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:54.528150 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:54.530867 (+ 2717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:54.530869 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:54.530869 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:54.530877 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:54.532282 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:54.532910 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:54.532911 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:54.532912 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:54.532980 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:54.533366 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:54.533367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:54.533368 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:54.533506 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:54.533573 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:54.534061 (+ 488us) server_negotiation.cc:299] Negotiation successful
0811 05:12:54.534151 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 05:12:54.534498 1523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:54.523134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39113 (local address 127.28.143.129:55461)
0811 05:12:54.523344 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:54.523483 (+ 139us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:54.524070 (+ 587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:54.524398 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:54.524403 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:54.525265 (+ 862us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:54.525576 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:54.525583 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:54.526443 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:54.526444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:54.527693 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:54.527697 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:54.528204 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:54.528206 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:54.528341 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:54.528967 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:12:54.528977 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:54.530727 (+ 1750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:54.532385 (+ 1658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:54.532397 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:54.532400 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:54.532781 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:54.533080 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:54.533081 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:54.533082 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:54.533244 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:54.533580 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:54.533587 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:54.533775 (+ 188us) client_negotiation.cc:769] Sending connection context
0811 05:12:54.533937 (+ 162us) client_negotiation.cc:240] Negotiation successful
0811 05:12:54.534229 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":108,"threads_started":1}
I20260811 05:12:54.535130 1520 heartbeater.cc:347] Connected to a master server at 127.28.143.190:39113
I20260811 05:12:54.535234 1520 heartbeater.cc:464] Registering TS with master...
I20260811 05:12:54.535413 1520 heartbeater.cc:511] Master 127.28.143.190:39113 requested a full tablet report, sending...
I20260811 05:12:54.536228 1181 ts_manager.cc:194] Registered new tserver with Master: 530eb154eba04f84a1be17f6fcff3947 (127.28.143.129:32985)
I20260811 05:12:54.536942 1181 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.129@KRBTEST.COM'} at 127.28.143.129:55461
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab.
Entry for principal kudu/127.28.143.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab.
I20260811 05:12:54.556734 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.130:0
--local_ip_for_outbound_sockets=127.28.143.130
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:39113
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:33247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:54.647394 1528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:54.647619 1528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:54.647644 1528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:54.648839 1528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:54.648877 1528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:54.648903 1528 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.130
I20260811 05:12:54.650367 1528 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:33247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:39113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1528
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.130
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:12:54.650571 1528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:54.650770 1528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:54.655939 1536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:54.655966 1539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:54.655936 1541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:54.656248 1528 server_base.cc:1034] running on GCE node
I20260811 05:12:54.656458 1528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:54.656730 1528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:54.657902 1528 hybrid_clock.cc:648] HybridClock initialized: now 1786425174657884 us; error 28 us; skew 500 ppm
Aug 11 05:12:54 dist-test-slave-f7th krb5kdc[1096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425174, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:54.660791 1528 init.cc:377] Logged in from keytab as kudu/127.28.143.130@KRBTEST.COM (short username kudu)
I20260811 05:12:54.661398 1528 webserver.cc:460] Webserver started at http://127.28.143.130:35061/ using document root <none> and password file <none>
I20260811 05:12:54.661672 1528 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:54.661728 1528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:54.661864 1528 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:54.662999 1528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-1/data/instance:
uuid: "6ad81b3218124f50a6ff330d45487c50"
format_stamp: "Formatted at 2026-08-11 05:12:54 on dist-test-slave-f7th"
server_key: "f165ec512c28f7a5b0a8211f12fd7900"
server_key_iv: "00b21c7f41aea019f33590fbde098298"
server_key_version: "encryptionkey@0"
I20260811 05:12:54.663363 1528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance:
uuid: "6ad81b3218124f50a6ff330d45487c50"
format_stamp: "Formatted at 2026-08-11 05:12:54 on dist-test-slave-f7th"
server_key: "f165ec512c28f7a5b0a8211f12fd7900"
server_key_iv: "00b21c7f41aea019f33590fbde098298"
server_key_version: "encryptionkey@0"
I20260811 05:12:54.664883 1528 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 05:12:54.665666 1557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:54.665855 1528 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:12:54.665942 1528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-1/wal
uuid: "6ad81b3218124f50a6ff330d45487c50"
format_stamp: "Formatted at 2026-08-11 05:12:54 on dist-test-slave-f7th"
server_key: "f165ec512c28f7a5b0a8211f12fd7900"
server_key_iv: "00b21c7f41aea019f33590fbde098298"
server_key_version: "encryptionkey@0"
I20260811 05:12:54.666007 1528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:54.683099 1528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:54.685460 1528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:54.685632 1528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:54.685892 1528 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:12:54.686210 1528 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:12:54.686245 1528 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:54.686276 1528 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:12:54.686293 1528 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:54.692028 1528 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.130:37933
I20260811 05:12:54.692101 1753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.130:37933 every 8 connection(s)
I20260811 05:12:54.692420 1528 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
I20260811 05:12:54.694234 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 1528
I20260811 05:12:54.694368 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance
I20260811 05:12:54.694628 29246 external_mini_cluster.cc:1428] Setting key db4fc67b0602dd8f9a820b3538d7532a
Aug 11 05:12:54 dist-test-slave-f7th krb5kdc[1096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425174, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:12:54.703361 1232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:54.693210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:50391 (local address 127.28.143.190:39113)
0811 05:12:54.693300 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:54.693302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:54.693565 (+ 263us) server_negotiation.cc:407] Connection header received
0811 05:12:54.694112 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:54.694114 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:54.694126 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:54.694187 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:54.695562 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:54.696137 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:54.697559 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:54.697765 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:54.697825 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:54.700515 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:54.700517 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:54.700517 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:54.700525 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:54.701922 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:54.702415 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:54.702416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:54.702416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:54.702468 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:54.702729 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:54.702729 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:54.702730 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:54.702833 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:54.702886 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:54.703215 (+ 329us) server_negotiation.cc:299] Negotiation successful
0811 05:12:54.703283 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 05:12:54.703644 1759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:54.693173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39113 (local address 127.28.143.130:50391)
0811 05:12:54.693383 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:54.693427 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:54.693949 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:54.694265 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:54.694269 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:54.695165 (+ 896us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:54.695435 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:54.695442 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:54.696226 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:54.696227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:54.697465 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:54.697469 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:54.697844 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:54.697845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:54.697988 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:54.698604 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:12:54.698615 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:54.700373 (+ 1758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:54.702012 (+ 1639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:54.702019 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:54.702022 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:54.702333 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:54.702526 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:54.702526 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:54.702527 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:54.702670 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:54.702888 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:54.702895 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:54.703079 (+ 184us) client_negotiation.cc:769] Sending connection context
0811 05:12:54.703206 (+ 127us) client_negotiation.cc:240] Negotiation successful
0811 05:12:54.703383 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":93,"threads_started":1}
I20260811 05:12:54.704264 1757 heartbeater.cc:347] Connected to a master server at 127.28.143.190:39113
I20260811 05:12:54.704392 1757 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.28.143.131@KRBTEST.COM; defaulting to no policy
I20260811 05:12:54.704636 1757 heartbeater.cc:511] Master 127.28.143.190:39113 requested a full tablet report, sending...
I20260811 05:12:54.705128 1181 ts_manager.cc:194] Registered new tserver with Master: 6ad81b3218124f50a6ff330d45487c50 (127.28.143.130:37933)
I20260811 05:12:54.705484 1181 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.130@KRBTEST.COM'} at 127.28.143.130:50391
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab.
Entry for principal kudu/127.28.143.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab.
I20260811 05:12:54.723730 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.131:0
--local_ip_for_outbound_sockets=127.28.143.131
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:39113
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:33247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:54.812103 1766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:54.812358 1766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:54.812386 1766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:54.813804 1766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:54.813866 1766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:54.813896 1766 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.131
I20260811 05:12:54.815547 1766 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:33247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:39113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1766
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.131
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:12:54.815801 1766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:54.816030 1766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:54.821529 1776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:54.821545 1773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:54.821547 1774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:54.822130 1766 server_base.cc:1034] running on GCE node
I20260811 05:12:54.822420 1766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:54.822693 1766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:54.823874 1766 hybrid_clock.cc:648] HybridClock initialized: now 1786425174823834 us; error 50 us; skew 500 ppm
Aug 11 05:12:54 dist-test-slave-f7th krb5kdc[1096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425174, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:54.827109 1766 init.cc:377] Logged in from keytab as kudu/127.28.143.131@KRBTEST.COM (short username kudu)
I20260811 05:12:54.827662 1766 webserver.cc:460] Webserver started at http://127.28.143.131:43077/ using document root <none> and password file <none>
I20260811 05:12:54.827916 1766 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:54.827976 1766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:54.828130 1766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:54.829444 1766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-2/data/instance:
uuid: "c883491764b6402382447265ce179394"
format_stamp: "Formatted at 2026-08-11 05:12:54 on dist-test-slave-f7th"
server_key: "2ae9d1fd8f320c3b0e2ff5c0a797d54e"
server_key_iv: "8bbf6be74e37b6e797d492fad9f1c391"
server_key_version: "encryptionkey@0"
I20260811 05:12:54.829873 1766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance:
uuid: "c883491764b6402382447265ce179394"
format_stamp: "Formatted at 2026-08-11 05:12:54 on dist-test-slave-f7th"
server_key: "2ae9d1fd8f320c3b0e2ff5c0a797d54e"
server_key_iv: "8bbf6be74e37b6e797d492fad9f1c391"
server_key_version: "encryptionkey@0"
I20260811 05:12:54.831698 1766 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:12:54.832796 1785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:54.832984 1766 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:12:54.833125 1766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-2/wal
uuid: "c883491764b6402382447265ce179394"
format_stamp: "Formatted at 2026-08-11 05:12:54 on dist-test-slave-f7th"
server_key: "2ae9d1fd8f320c3b0e2ff5c0a797d54e"
server_key_iv: "8bbf6be74e37b6e797d492fad9f1c391"
server_key_version: "encryptionkey@0"
I20260811 05:12:54.833207 1766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:54.869815 1766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:54.872150 1766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:54.872304 1766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:54.872548 1766 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:12:54.872895 1766 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:12:54.872931 1766 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:54.872962 1766 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:12:54.872979 1766 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:54.878679 1766 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.131:40039
I20260811 05:12:54.878747 1981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.131:40039 every 8 connection(s)
I20260811 05:12:54.879077 1766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
I20260811 05:12:54.880889 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 1766
I20260811 05:12:54.880991 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance
I20260811 05:12:54.881201 29246 external_mini_cluster.cc:1428] Setting key 00c3fbd7a51826112405dfea8dbdff64
Aug 11 05:12:54 dist-test-slave-f7th krb5kdc[1096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425174, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:12:54.889748 1232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:54.880223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:41689 (local address 127.28.143.190:39113)
0811 05:12:54.880319 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:54.880321 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:54.880496 (+ 175us) server_negotiation.cc:407] Connection header received
0811 05:12:54.881145 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:54.881147 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:54.881162 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:54.881236 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:54.882446 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:54.882945 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:54.884072 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:54.884272 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:54.884335 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:54.886744 (+ 2409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:54.886746 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:54.886746 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:54.886755 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:54.888134 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:54.888602 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:54.888604 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:54.888604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:54.888703 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:54.889017 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:54.889019 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:54.889019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:54.889166 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:54.889225 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:54.889564 (+ 339us) server_negotiation.cc:299] Negotiation successful
0811 05:12:54.889649 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 05:12:54.889863 1988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:54.879888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39113 (local address 127.28.143.131:41689)
0811 05:12:54.880265 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:54.880321 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:54.880884 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:54.881329 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:54.881333 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:54.882089 (+ 756us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:54.882361 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:54.882368 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:54.883079 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:54.883080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:54.883932 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:54.883936 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:54.884345 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:54.884345 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:54.884454 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:54.885037 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:12:54.885046 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:54.886597 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:54.888231 (+ 1634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:54.888238 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:54.888240 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:54.888482 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:54.888783 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:54.888784 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:54.888784 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:54.888910 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:54.889249 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:54.889256 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:54.889401 (+ 145us) client_negotiation.cc:769] Sending connection context
0811 05:12:54.889503 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 05:12:54.889651 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":133,"threads_started":1}
I20260811 05:12:54.890432 1982 heartbeater.cc:347] Connected to a master server at 127.28.143.190:39113
I20260811 05:12:54.890549 1982 heartbeater.cc:464] Registering TS with master...
I20260811 05:12:54.890748 1982 heartbeater.cc:511] Master 127.28.143.190:39113 requested a full tablet report, sending...
I20260811 05:12:54.891294 1181 ts_manager.cc:194] Registered new tserver with Master: c883491764b6402382447265ce179394 (127.28.143.131:40039)
I20260811 05:12:54.891723 1181 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.131@KRBTEST.COM'} at 127.28.143.131:41689
I20260811 05:12:54.893510 29246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 05:12:54.907601 1232 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:12:54.906591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53104 (local address 127.28.143.190:39113)
0811 05:12:54.906683 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:54.906686 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:54.906784 (+ 98us) server_negotiation.cc:407] Connection header received
0811 05:12:54.906873 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:54.906874 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:54.906886 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:54.906930 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:54.907525 (+ 595us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53104: BlockingRecv error: recv got EOF from 127.0.0.1:53104 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 05:12:54.908448 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 1283
I20260811 05:12:54.939105 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 1528
I20260811 05:12:54.954177 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 1766
I20260811 05:12:54.974074 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 1115
2026-08-11T05:12:54Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2198 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425159060450-29246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:12:55 dist-test-slave-f7th krb5kdc[2010](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:12:55 dist-test-slave-f7th krb5kdc[2010](info): set up 2 sockets
Aug 11 05:12:55 dist-test-slave-f7th krb5kdc[2010](info): commencing operation
krb5kdc: starting...
W20260811 05:12:56.216105 29246 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Aug 11 05:12:56 dist-test-slave-f7th krb5kdc[2010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425176, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:12:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:12:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
I20260811 05:12:56.356174 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:46421
--webserver_interface=127.28.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:38667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:46421
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:56.442866 2028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:56.443120 2028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:56.443154 2028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:56.444433 2028 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:12:56.444487 2028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:56.444506 2028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:56.444523 2028 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:12:56.444538 2028 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:12:56.446300 2028 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:38667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:46421
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:46421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2028
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:12:56.446604 2028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:56.446859 2028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:56.451843 2046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:56.451848 2050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:56.452105 2028 server_base.cc:1034] running on GCE node
W20260811 05:12:56.451946 2047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:56.452453 2028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:56.452915 2028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:56.454102 2028 hybrid_clock.cc:648] HybridClock initialized: now 1786425176454062 us; error 55 us; skew 500 ppm
Aug 11 05:12:56 dist-test-slave-f7th krb5kdc[2010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425176, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:56.462599 2028 init.cc:377] Logged in from keytab as kudu/127.28.143.190@KRBTEST.COM (short username kudu)
I20260811 05:12:56.463156 2028 webserver.cc:460] Webserver started at http://127.28.143.190:44765/ using document root <none> and password file <none>
I20260811 05:12:56.463440 2028 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:56.463500 2028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:56.463637 2028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:56.464828 2028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "54b1a689f6fa48a5a110995e2ec1fb42"
format_stamp: "Formatted at 2026-08-11 05:12:56 on dist-test-slave-f7th"
server_key: "a8cc9752645c50ea3c9b582ac6cc950c"
server_key_iv: "efb6d902e6340a1dfd474ec4d0a1acae"
server_key_version: "encryptionkey@0"
I20260811 05:12:56.465183 2028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "54b1a689f6fa48a5a110995e2ec1fb42"
format_stamp: "Formatted at 2026-08-11 05:12:56 on dist-test-slave-f7th"
server_key: "a8cc9752645c50ea3c9b582ac6cc950c"
server_key_iv: "efb6d902e6340a1dfd474ec4d0a1acae"
server_key_version: "encryptionkey@0"
I20260811 05:12:56.466588 2028 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:12:56.467411 2058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:56.467636 2028 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:12:56.467741 2028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "54b1a689f6fa48a5a110995e2ec1fb42"
format_stamp: "Formatted at 2026-08-11 05:12:56 on dist-test-slave-f7th"
server_key: "a8cc9752645c50ea3c9b582ac6cc950c"
server_key_iv: "efb6d902e6340a1dfd474ec4d0a1acae"
server_key_version: "encryptionkey@0"
I20260811 05:12:56.467808 2028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:56.494594 2028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:56.497442 2028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:56.497622 2028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:56.502024 2028 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:46421
I20260811 05:12:56.502072 2164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:46421 every 8 connection(s)
I20260811 05:12:56.502655 2028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:12:56.503455 2166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:56.505949 2166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54b1a689f6fa48a5a110995e2ec1fb42: Bootstrap starting.
I20260811 05:12:56.506594 2166 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54b1a689f6fa48a5a110995e2ec1fb42: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:56.506928 2166 log.cc:824] T 00000000000000000000000000000000 P 54b1a689f6fa48a5a110995e2ec1fb42: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:56.507861 2166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54b1a689f6fa48a5a110995e2ec1fb42: No bootstrap required, opened a new log
I20260811 05:12:56.509613 2166 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54b1a689f6fa48a5a110995e2ec1fb42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54b1a689f6fa48a5a110995e2ec1fb42" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 46421 } }
I20260811 05:12:56.509768 2166 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54b1a689f6fa48a5a110995e2ec1fb42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:56.509790 2166 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54b1a689f6fa48a5a110995e2ec1fb42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54b1a689f6fa48a5a110995e2ec1fb42, State: Initialized, Role: FOLLOWER
I20260811 05:12:56.509928 2166 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54b1a689f6fa48a5a110995e2ec1fb42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54b1a689f6fa48a5a110995e2ec1fb42" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 46421 } }
I20260811 05:12:56.510004 2166 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54b1a689f6fa48a5a110995e2ec1fb42 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:12:56.510030 2166 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54b1a689f6fa48a5a110995e2ec1fb42 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:12:56.510066 2166 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54b1a689f6fa48a5a110995e2ec1fb42 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:56.511056 2166 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54b1a689f6fa48a5a110995e2ec1fb42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54b1a689f6fa48a5a110995e2ec1fb42" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 46421 } }
I20260811 05:12:56.511159 2166 leader_election.cc:302] T 00000000000000000000000000000000 P 54b1a689f6fa48a5a110995e2ec1fb42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54b1a689f6fa48a5a110995e2ec1fb42; no voters:
I20260811 05:12:56.511338 2166 leader_election.cc:288] T 00000000000000000000000000000000 P 54b1a689f6fa48a5a110995e2ec1fb42 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:12:56.511417 2169 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54b1a689f6fa48a5a110995e2ec1fb42 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:56.511617 2166 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54b1a689f6fa48a5a110995e2ec1fb42 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:12:56.511629 2169 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54b1a689f6fa48a5a110995e2ec1fb42 [term 1 LEADER]: Becoming Leader. State: Replica: 54b1a689f6fa48a5a110995e2ec1fb42, State: Running, Role: LEADER
I20260811 05:12:56.511827 2169 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54b1a689f6fa48a5a110995e2ec1fb42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54b1a689f6fa48a5a110995e2ec1fb42" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 46421 } }
I20260811 05:12:56.512406 2174 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54b1a689f6fa48a5a110995e2ec1fb42 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54b1a689f6fa48a5a110995e2ec1fb42. Latest consensus state: current_term: 1 leader_uuid: "54b1a689f6fa48a5a110995e2ec1fb42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54b1a689f6fa48a5a110995e2ec1fb42" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 46421 } } }
I20260811 05:12:56.512414 2173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54b1a689f6fa48a5a110995e2ec1fb42 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54b1a689f6fa48a5a110995e2ec1fb42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54b1a689f6fa48a5a110995e2ec1fb42" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 46421 } } }
I20260811 05:12:56.512504 2174 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54b1a689f6fa48a5a110995e2ec1fb42 [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:56.512522 2173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54b1a689f6fa48a5a110995e2ec1fb42 [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:56.512724 2188 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:12:56.512893 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 2028
I20260811 05:12:56.512990 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:12:56.513190 29246 external_mini_cluster.cc:1428] Setting key 82e6bd784e767ac016b17200ece6bf26
I20260811 05:12:56.513290 2188 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:12:56.514947 2188 catalog_manager.cc:1347] Generated new cluster ID: 5456c1ddc3b44f93b0cd15829ddeca9f
I20260811 05:12:56.515017 2188 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 05:12:56 dist-test-slave-f7th krb5kdc[2010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425176, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:12:56.519702 2188 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:12:56.520540 2188 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:12:56.523016 2204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:56.513663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35012 (local address 127.28.143.190:46421)
0811 05:12:56.513869 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:56.513879 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:56.513921 (+ 42us) server_negotiation.cc:407] Connection header received
0811 05:12:56.514072 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:56.514078 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:56.514123 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:56.514237 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:56.514972 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:56.515932 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:56.516711 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:56.517169 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:56.517269 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:56.519889 (+ 2620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:56.519896 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:56.519896 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:56.519911 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:56.521570 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:56.521921 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:56.521925 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:56.521925 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:56.522012 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:56.522191 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:56.522192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:56.522193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:56.522415 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:56.522586 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:56.522656 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 05:12:56.522814 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":79,"threads_started":1}
I20260811 05:12:56.523921 29246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:12:56.527607 2188 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54b1a689f6fa48a5a110995e2ec1fb42: Generated new TSK 0
I20260811 05:12:56.527805 2188 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:12:56.540251 2204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:56.536081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35016 (local address 127.28.143.190:46421)
0811 05:12:56.536139 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:56.536142 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:56.536192 (+ 50us) server_negotiation.cc:407] Connection header received
0811 05:12:56.536256 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:56.536257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:56.536281 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:56.536364 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:56.536888 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:56.537711 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:56.538527 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:56.538899 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:56.538982 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:56.539683 (+ 701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:56.539685 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:56.539685 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:12:56.539954 (+ 269us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:56.539975 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:12:56.539983 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:56.540042 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:56.540123 (+ 81us) server_negotiation.cc:299] Negotiation successful
0811 05:12:56.540158 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 05:12:56.542824 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 2028
2026-08-11T05:12:56Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1562 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:12:56 dist-test-slave-f7th krb5kdc[2231](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:12:56 dist-test-slave-f7th krb5kdc[2231](info): set up 2 sockets
Aug 11 05:12:56 dist-test-slave-f7th krb5kdc[2231](info): commencing operation
krb5kdc: starting...
W20260811 05:12:57.781272 29246 mini_kdc.cc:121] Time spent starting KDC: real 1.214s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Aug 11 05:12:57 dist-test-slave-f7th krb5kdc[2231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425177, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:12:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:12:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
I20260811 05:12:57.920114 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:45015
--webserver_interface=127.28.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:46539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:45015
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:58.009974 2251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:58.010246 2251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:58.010274 2251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:58.011857 2251 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:12:58.012054 2251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:58.012081 2251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:58.012097 2251 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:12:58.012248 2251 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:12:58.014578 2251 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:46539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:45015
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:45015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2251
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:12:58.014878 2251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:58.015199 2251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:58.021318 2265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:58.021320 2259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:58.021344 2251 server_base.cc:1034] running on GCE node
W20260811 05:12:58.021371 2261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:58.021932 2251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:58.022284 2251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:58.023501 2251 hybrid_clock.cc:648] HybridClock initialized: now 1786425178023488 us; error 113 us; skew 500 ppm
Aug 11 05:12:58 dist-test-slave-f7th krb5kdc[2231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425178, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:58.027240 2251 init.cc:377] Logged in from keytab as kudu/127.28.143.190@KRBTEST.COM (short username kudu)
I20260811 05:12:58.027844 2251 webserver.cc:460] Webserver started at http://127.28.143.190:45799/ using document root <none> and password file <none>
I20260811 05:12:58.028123 2251 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:58.028180 2251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:58.028322 2251 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:58.029543 2251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "a59dd1f4786d4576bc5a5e3d42c0d373"
format_stamp: "Formatted at 2026-08-11 05:12:58 on dist-test-slave-f7th"
server_key: "60e9c5c2bb988fde681a51a8496165ed"
server_key_iv: "dbbabde17b57c1167a0550d59fcc499d"
server_key_version: "encryptionkey@0"
I20260811 05:12:58.029927 2251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "a59dd1f4786d4576bc5a5e3d42c0d373"
format_stamp: "Formatted at 2026-08-11 05:12:58 on dist-test-slave-f7th"
server_key: "60e9c5c2bb988fde681a51a8496165ed"
server_key_iv: "dbbabde17b57c1167a0550d59fcc499d"
server_key_version: "encryptionkey@0"
I20260811 05:12:58.031631 2251 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260811 05:12:58.032672 2273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:58.032860 2251 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 05:12:58.033010 2251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "a59dd1f4786d4576bc5a5e3d42c0d373"
format_stamp: "Formatted at 2026-08-11 05:12:58 on dist-test-slave-f7th"
server_key: "60e9c5c2bb988fde681a51a8496165ed"
server_key_iv: "dbbabde17b57c1167a0550d59fcc499d"
server_key_version: "encryptionkey@0"
I20260811 05:12:58.033089 2251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:58.055092 2251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:58.057801 2251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:58.057979 2251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:58.062177 2251 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:45015
I20260811 05:12:58.062233 2364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:45015 every 8 connection(s)
I20260811 05:12:58.062610 2251 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:12:58.063781 2365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:12:58.066424 2365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a59dd1f4786d4576bc5a5e3d42c0d373: Bootstrap starting.
I20260811 05:12:58.066929 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 2251
I20260811 05:12:58.067054 2365 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a59dd1f4786d4576bc5a5e3d42c0d373: Neither blocks nor log segments found. Creating new log.
I20260811 05:12:58.067067 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:12:58.067245 29246 external_mini_cluster.cc:1428] Setting key 4ac3efe891b2a5f442307b82634b4fc7
I20260811 05:12:58.067375 2365 log.cc:824] T 00000000000000000000000000000000 P a59dd1f4786d4576bc5a5e3d42c0d373: Log is configured to *not* fsync() on all Append() calls
I20260811 05:12:58.068312 2365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a59dd1f4786d4576bc5a5e3d42c0d373: No bootstrap required, opened a new log
I20260811 05:12:58.070818 2365 raft_consensus.cc:348] T 00000000000000000000000000000000 P a59dd1f4786d4576bc5a5e3d42c0d373 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a59dd1f4786d4576bc5a5e3d42c0d373" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 45015 } }
I20260811 05:12:58.071024 2365 raft_consensus.cc:374] T 00000000000000000000000000000000 P a59dd1f4786d4576bc5a5e3d42c0d373 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:12:58.071059 2365 raft_consensus.cc:729] T 00000000000000000000000000000000 P a59dd1f4786d4576bc5a5e3d42c0d373 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a59dd1f4786d4576bc5a5e3d42c0d373, State: Initialized, Role: FOLLOWER
I20260811 05:12:58.071242 2365 consensus_queue.cc:260] T 00000000000000000000000000000000 P a59dd1f4786d4576bc5a5e3d42c0d373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a59dd1f4786d4576bc5a5e3d42c0d373" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 45015 } }
I20260811 05:12:58.071380 2365 raft_consensus.cc:388] T 00000000000000000000000000000000 P a59dd1f4786d4576bc5a5e3d42c0d373 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:12:58.071453 2365 raft_consensus.cc:482] T 00000000000000000000000000000000 P a59dd1f4786d4576bc5a5e3d42c0d373 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:12:58.071522 2365 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a59dd1f4786d4576bc5a5e3d42c0d373 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:12:58.072386 2365 raft_consensus.cc:504] T 00000000000000000000000000000000 P a59dd1f4786d4576bc5a5e3d42c0d373 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a59dd1f4786d4576bc5a5e3d42c0d373" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 45015 } }
I20260811 05:12:58.072544 2365 leader_election.cc:302] T 00000000000000000000000000000000 P a59dd1f4786d4576bc5a5e3d42c0d373 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a59dd1f4786d4576bc5a5e3d42c0d373; no voters:
I20260811 05:12:58.072762 2365 leader_election.cc:288] T 00000000000000000000000000000000 P a59dd1f4786d4576bc5a5e3d42c0d373 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:12:58.073065 2365 sys_catalog.cc:564] T 00000000000000000000000000000000 P a59dd1f4786d4576bc5a5e3d42c0d373 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 05:12:58.075271 2395 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a59dd1f4786d4576bc5a5e3d42c0d373: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:12:58.075354 2395 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 11 05:12:58 dist-test-slave-f7th krb5kdc[2231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425177, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:12:58.080078 2380 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a59dd1f4786d4576bc5a5e3d42c0d373 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:12:58.080310 2380 raft_consensus.cc:686] T 00000000000000000000000000000000 P a59dd1f4786d4576bc5a5e3d42c0d373 [term 1 LEADER]: Becoming Leader. State: Replica: a59dd1f4786d4576bc5a5e3d42c0d373, State: Running, Role: LEADER
I20260811 05:12:58.080438 2380 consensus_queue.cc:237] T 00000000000000000000000000000000 P a59dd1f4786d4576bc5a5e3d42c0d373 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a59dd1f4786d4576bc5a5e3d42c0d373" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 45015 } }
I20260811 05:12:58.080832 2380 sys_catalog.cc:455] T 00000000000000000000000000000000 P a59dd1f4786d4576bc5a5e3d42c0d373 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a59dd1f4786d4576bc5a5e3d42c0d373" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a59dd1f4786d4576bc5a5e3d42c0d373" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 45015 } } }
I20260811 05:12:58.080950 2380 sys_catalog.cc:458] T 00000000000000000000000000000000 P a59dd1f4786d4576bc5a5e3d42c0d373 [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:58.081099 2380 sys_catalog.cc:455] T 00000000000000000000000000000000 P a59dd1f4786d4576bc5a5e3d42c0d373 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a59dd1f4786d4576bc5a5e3d42c0d373. Latest consensus state: current_term: 1 leader_uuid: "a59dd1f4786d4576bc5a5e3d42c0d373" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a59dd1f4786d4576bc5a5e3d42c0d373" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 45015 } } }
I20260811 05:12:58.081126 2376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:58.067765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48424 (local address 127.28.143.190:45015)
0811 05:12:58.067964 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:58.067975 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:58.068018 (+ 43us) server_negotiation.cc:407] Connection header received
0811 05:12:58.068669 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:58.068681 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:58.068757 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:58.068940 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:58.069621 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:58.071060 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:58.072172 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:58.072853 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:58.072981 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:58.076380 (+ 3399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:58.076385 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:58.076386 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:58.076400 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:58.078606 (+ 2206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:58.079111 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:58.079115 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:58.079116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:58.079241 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:58.079457 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:58.079458 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:58.079459 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:58.079853 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:58.080023 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:58.080605 (+ 582us) server_negotiation.cc:299] Negotiation successful
0811 05:12:58.080812 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":93,"threads_started":1}
I20260811 05:12:58.081161 2380 sys_catalog.cc:458] T 00000000000000000000000000000000 P a59dd1f4786d4576bc5a5e3d42c0d373 [sys.catalog]: This master's current role is: LEADER
I20260811 05:12:58.082458 2403 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:12:58.082687 2403 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:12:58.084525 2403 catalog_manager.cc:1347] Generated new cluster ID: 1bc56ffb422040ed827092b61908b1a8
I20260811 05:12:58.084587 2403 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:12:58.093712 2403 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:12:58.094307 2403 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:12:58.097391 2403 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a59dd1f4786d4576bc5a5e3d42c0d373: Generated new TSK 0
I20260811 05:12:58.097692 2403 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.143.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab.
Entry for principal kudu/127.28.143.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab.
I20260811 05:12:58.133947 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.129:0
--local_ip_for_outbound_sockets=127.28.143.129
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:45015
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:46539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:58.223649 2411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:58.223897 2411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:58.223932 2411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:58.225438 2411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:58.225512 2411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:58.225555 2411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.129
I20260811 05:12:58.227303 2411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:46539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:45015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.129
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:12:58.227598 2411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:58.227849 2411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:58.233462 2422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:58.233465 2418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:58.233687 2420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:58.233680 2411 server_base.cc:1034] running on GCE node
I20260811 05:12:58.233932 2411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:58.234261 2411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:58.235522 2411 hybrid_clock.cc:648] HybridClock initialized: now 1786425178235480 us; error 36 us; skew 500 ppm
Aug 11 05:12:58 dist-test-slave-f7th krb5kdc[2231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425178, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:58.238584 2411 init.cc:377] Logged in from keytab as kudu/127.28.143.129@KRBTEST.COM (short username kudu)
I20260811 05:12:58.239128 2411 webserver.cc:460] Webserver started at http://127.28.143.129:42345/ using document root <none> and password file <none>
I20260811 05:12:58.239346 2411 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:58.239399 2411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:58.239535 2411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:58.240675 2411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-0/data/instance:
uuid: "c8d967778f3b4b71ac6c3218d08b12c6"
format_stamp: "Formatted at 2026-08-11 05:12:58 on dist-test-slave-f7th"
server_key: "c372f5233abe2f9369f3804898de9352"
server_key_iv: "d42164d181b925d6d5283ef750f73419"
server_key_version: "encryptionkey@0"
I20260811 05:12:58.241042 2411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance:
uuid: "c8d967778f3b4b71ac6c3218d08b12c6"
format_stamp: "Formatted at 2026-08-11 05:12:58 on dist-test-slave-f7th"
server_key: "c372f5233abe2f9369f3804898de9352"
server_key_iv: "d42164d181b925d6d5283ef750f73419"
server_key_version: "encryptionkey@0"
I20260811 05:12:58.242427 2411 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 05:12:58.243261 2435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:58.243494 2411 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 05:12:58.243685 2411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-0/wal
uuid: "c8d967778f3b4b71ac6c3218d08b12c6"
format_stamp: "Formatted at 2026-08-11 05:12:58 on dist-test-slave-f7th"
server_key: "c372f5233abe2f9369f3804898de9352"
server_key_iv: "d42164d181b925d6d5283ef750f73419"
server_key_version: "encryptionkey@0"
I20260811 05:12:58.243806 2411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:58.258195 2411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:58.260756 2411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:58.260926 2411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:58.261279 2411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:12:58.261687 2411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:12:58.261727 2411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:58.261759 2411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:12:58.261776 2411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:58.267686 2411 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.129:45227
I20260811 05:12:58.267786 2651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.129:45227 every 8 connection(s)
I20260811 05:12:58.268096 2411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
I20260811 05:12:58.271426 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 2411
I20260811 05:12:58.271517 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance
I20260811 05:12:58.271694 29246 external_mini_cluster.cc:1428] Setting key e958df09109405b943d9aa62b2f4b978
Aug 11 05:12:58 dist-test-slave-f7th krb5kdc[2231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425178, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:12:58.278633 2376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:58.268915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:50359 (local address 127.28.143.190:45015)
0811 05:12:58.268966 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:58.268968 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:58.269180 (+ 212us) server_negotiation.cc:407] Connection header received
0811 05:12:58.269649 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:58.269650 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:58.269660 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:58.269719 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:58.270638 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:58.271148 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:58.272445 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:58.272744 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:58.272831 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:58.275365 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:58.275366 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:58.275367 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:58.275375 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:58.276952 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:58.277402 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:58.277404 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:58.277404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:58.277482 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:58.277884 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:58.277886 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:58.277886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:58.278016 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:58.278136 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:58.278418 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 05:12:58.278527 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 05:12:58.278805 2655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:58.268892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:45015 (local address 127.28.143.129:50359)
0811 05:12:58.269052 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:58.269097 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:58.269571 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:58.269774 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:58.269778 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:58.270363 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:58.270563 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:58.270568 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:58.271235 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:58.271236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:58.272331 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:58.272334 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:58.272869 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:58.272870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:58.273040 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:58.273588 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:12:58.273598 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:58.275227 (+ 1629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:58.277057 (+ 1830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:58.277066 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:58.277069 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:58.277320 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:58.277597 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:58.277598 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:58.277598 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:58.277787 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:58.278135 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:58.278144 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:58.278288 (+ 144us) client_negotiation.cc:769] Sending connection context
0811 05:12:58.278383 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 05:12:58.278573 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":66,"threads_started":1}
I20260811 05:12:58.279335 2653 heartbeater.cc:347] Connected to a master server at 127.28.143.190:45015
I20260811 05:12:58.279434 2653 heartbeater.cc:464] Registering TS with master...
I20260811 05:12:58.279623 2653 heartbeater.cc:511] Master 127.28.143.190:45015 requested a full tablet report, sending...
I20260811 05:12:58.280203 2298 ts_manager.cc:194] Registered new tserver with Master: c8d967778f3b4b71ac6c3218d08b12c6 (127.28.143.129:45227)
I20260811 05:12:58.280889 2298 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.129@KRBTEST.COM'} at 127.28.143.129:50359
WARNING: no policy specified for kudu/127.28.143.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab.
Entry for principal kudu/127.28.143.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab.
I20260811 05:12:58.305080 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.130:0
--local_ip_for_outbound_sockets=127.28.143.130
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:45015
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:46539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:58.390741 2658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:58.390982 2658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:58.391009 2658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:58.392180 2658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:58.392225 2658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:58.392251 2658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.130
I20260811 05:12:58.393800 2658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:46539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:45015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.130
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:12:58.394033 2658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:58.394238 2658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:58.399581 2675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:58.399736 2658 server_base.cc:1034] running on GCE node
W20260811 05:12:58.399647 2669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:58.400185 2667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:58.400544 2658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:58.400876 2658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:58.402065 2658 hybrid_clock.cc:648] HybridClock initialized: now 1786425178402049 us; error 32 us; skew 500 ppm
Aug 11 05:12:58 dist-test-slave-f7th krb5kdc[2231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425178, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:58.404958 2658 init.cc:377] Logged in from keytab as kudu/127.28.143.130@KRBTEST.COM (short username kudu)
I20260811 05:12:58.405496 2658 webserver.cc:460] Webserver started at http://127.28.143.130:43975/ using document root <none> and password file <none>
I20260811 05:12:58.405740 2658 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:58.405792 2658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:58.405925 2658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:58.406991 2658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-1/data/instance:
uuid: "825a400ebcdc4753881d692ed5b802a2"
format_stamp: "Formatted at 2026-08-11 05:12:58 on dist-test-slave-f7th"
server_key: "e6082c498ef2c621cd79c66eac43b9e7"
server_key_iv: "124abdb703c771638d44c5f7f9acef47"
server_key_version: "encryptionkey@0"
I20260811 05:12:58.407326 2658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance:
uuid: "825a400ebcdc4753881d692ed5b802a2"
format_stamp: "Formatted at 2026-08-11 05:12:58 on dist-test-slave-f7th"
server_key: "e6082c498ef2c621cd79c66eac43b9e7"
server_key_iv: "124abdb703c771638d44c5f7f9acef47"
server_key_version: "encryptionkey@0"
I20260811 05:12:58.408756 2658 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 05:12:58.409566 2695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:58.409747 2658 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:12:58.409852 2658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-1/wal
uuid: "825a400ebcdc4753881d692ed5b802a2"
format_stamp: "Formatted at 2026-08-11 05:12:58 on dist-test-slave-f7th"
server_key: "e6082c498ef2c621cd79c66eac43b9e7"
server_key_iv: "124abdb703c771638d44c5f7f9acef47"
server_key_version: "encryptionkey@0"
I20260811 05:12:58.409931 2658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:58.420058 2658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:58.422654 2658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:58.422816 2658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:58.423059 2658 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:12:58.423388 2658 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:12:58.423424 2658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:58.423455 2658 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:12:58.423475 2658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:58.429945 2658 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.130:37537
I20260811 05:12:58.429991 2897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.130:37537 every 8 connection(s)
I20260811 05:12:58.430351 2658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
I20260811 05:12:58.431631 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 2658
I20260811 05:12:58.431726 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance
I20260811 05:12:58.431900 29246 external_mini_cluster.cc:1428] Setting key cc220663a4d8ec0be753ec44866993cd
Aug 11 05:12:58 dist-test-slave-f7th krb5kdc[2231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425178, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.28.143.131@KRBTEST.COM; defaulting to no policy
I20260811 05:12:58.441699 2376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:58.431127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:33257 (local address 127.28.143.190:45015)
0811 05:12:58.431189 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:58.431191 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:58.431412 (+ 221us) server_negotiation.cc:407] Connection header received
0811 05:12:58.432211 (+ 799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:58.432212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:58.432224 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:58.432288 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:58.433388 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:58.433989 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:58.435526 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:58.435776 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:58.435836 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:58.438639 (+ 2803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:58.438641 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:58.438641 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:58.438649 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:58.440057 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:58.440583 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:58.440584 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:58.440584 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:58.440670 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:58.440971 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:58.440971 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:58.440972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:58.441098 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:58.441154 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:58.441494 (+ 340us) server_negotiation.cc:299] Negotiation successful
0811 05:12:58.441614 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 05:12:58.441911 2908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:58.431081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:45015 (local address 127.28.143.130:33257)
0811 05:12:58.431243 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:58.431288 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:58.431988 (+ 700us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:58.432389 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:58.432394 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:58.433089 (+ 695us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:58.433291 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:58.433296 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:58.434129 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:58.434130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:58.435415 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:58.435419 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:58.435861 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:58.435863 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:58.436027 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:58.436671 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:12:58.436683 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:58.438507 (+ 1824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:58.440151 (+ 1644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:58.440162 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:58.440165 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:58.440472 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:58.440740 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:58.440741 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:58.440741 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:58.440894 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:58.441166 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:58.441174 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:58.441358 (+ 184us) client_negotiation.cc:769] Sending connection context
0811 05:12:58.441483 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 05:12:58.441670 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":74,"threads_started":1}
I20260811 05:12:58.442619 2898 heartbeater.cc:347] Connected to a master server at 127.28.143.190:45015
I20260811 05:12:58.442730 2898 heartbeater.cc:464] Registering TS with master...
I20260811 05:12:58.442943 2898 heartbeater.cc:511] Master 127.28.143.190:45015 requested a full tablet report, sending...
I20260811 05:12:58.443387 2298 ts_manager.cc:194] Registered new tserver with Master: 825a400ebcdc4753881d692ed5b802a2 (127.28.143.130:37537)
I20260811 05:12:58.443799 2298 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.130@KRBTEST.COM'} at 127.28.143.130:33257
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab.
Entry for principal kudu/127.28.143.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab.
I20260811 05:12:58.461838 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.131:0
--local_ip_for_outbound_sockets=127.28.143.131
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:45015
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:46539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:12:58.546432 2913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:12:58.546662 2913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:12:58.546697 2913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:12:58.547927 2913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:12:58.547978 2913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:12:58.548017 2913 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.131
I20260811 05:12:58.549636 2913 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:46539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:45015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.131
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:12:58.550016 2913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:12:58.550280 2913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:12:58.555451 2928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:58.555557 2913 server_base.cc:1034] running on GCE node
W20260811 05:12:58.555467 2924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:12:58.555462 2926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:12:58.555872 2913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:12:58.556126 2913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:12:58.557269 2913 hybrid_clock.cc:648] HybridClock initialized: now 1786425178557256 us; error 28 us; skew 500 ppm
Aug 11 05:12:58 dist-test-slave-f7th krb5kdc[2231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425178, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:12:58.559919 2913 init.cc:377] Logged in from keytab as kudu/127.28.143.131@KRBTEST.COM (short username kudu)
I20260811 05:12:58.560354 2913 webserver.cc:460] Webserver started at http://127.28.143.131:35963/ using document root <none> and password file <none>
I20260811 05:12:58.560571 2913 fs_manager.cc:359] Metadata directory not provided
I20260811 05:12:58.560639 2913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:12:58.560763 2913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:12:58.561802 2913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-2/data/instance:
uuid: "6211950da8eb4ec7822999779f957752"
format_stamp: "Formatted at 2026-08-11 05:12:58 on dist-test-slave-f7th"
server_key: "e26ea45961b759a2bc102293dcf88087"
server_key_iv: "1c71280440ee5ca8e4645f3d6cceef9d"
server_key_version: "encryptionkey@0"
I20260811 05:12:58.562107 2913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance:
uuid: "6211950da8eb4ec7822999779f957752"
format_stamp: "Formatted at 2026-08-11 05:12:58 on dist-test-slave-f7th"
server_key: "e26ea45961b759a2bc102293dcf88087"
server_key_iv: "1c71280440ee5ca8e4645f3d6cceef9d"
server_key_version: "encryptionkey@0"
I20260811 05:12:58.563433 2913 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 05:12:58.564283 2941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:58.564514 2913 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 05:12:58.564656 2913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-2/wal
uuid: "6211950da8eb4ec7822999779f957752"
format_stamp: "Formatted at 2026-08-11 05:12:58 on dist-test-slave-f7th"
server_key: "e26ea45961b759a2bc102293dcf88087"
server_key_iv: "1c71280440ee5ca8e4645f3d6cceef9d"
server_key_version: "encryptionkey@0"
I20260811 05:12:58.564728 2913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:12:58.573716 2913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:12:58.575938 2913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:12:58.576084 2913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:12:58.576309 2913 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:12:58.576699 2913 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:12:58.576738 2913 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:58.576771 2913 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:12:58.576793 2913 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:12:58.582813 2913 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.131:33079
I20260811 05:12:58.582935 3132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.131:33079 every 8 connection(s)
I20260811 05:12:58.583199 2913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
I20260811 05:12:58.588094 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 2913
I20260811 05:12:58.588177 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance
I20260811 05:12:58.588349 29246 external_mini_cluster.cc:1428] Setting key c8448e734b9d7388963a08b9f6d2aaad
Aug 11 05:12:58 dist-test-slave-f7th krb5kdc[2231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425178, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:12:58.593169 2376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:58.583943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:47003 (local address 127.28.143.190:45015)
0811 05:12:58.584010 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:58.584012 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:58.584205 (+ 193us) server_negotiation.cc:407] Connection header received
0811 05:12:58.584705 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:58.584707 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:58.584719 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:58.584788 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:58.585694 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:58.586204 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:58.587232 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:58.587451 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:58.587526 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:58.589971 (+ 2445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:58.589972 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:58.589972 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:58.589980 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:58.591490 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:58.591958 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:58.591960 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:58.591960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:58.592027 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:58.592423 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:58.592425 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:58.592425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:58.592559 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:58.592648 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:58.593004 (+ 356us) server_negotiation.cc:299] Negotiation successful
0811 05:12:58.593098 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 05:12:58.593292 3140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:58.583899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:45015 (local address 127.28.143.131:47003)
0811 05:12:58.584053 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 05:12:58.584096 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 05:12:58.584575 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:12:58.584841 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:12:58.584845 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:12:58.585405 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:12:58.585613 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:58.585618 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:58.586276 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:58.586277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:58.587134 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:12:58.587137 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:12:58.587554 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:12:58.587555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:12:58.587670 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:58.588227 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:12:58.588238 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:12:58.589864 (+ 1626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:12:58.591595 (+ 1731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:58.591602 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:58.591605 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:58.591867 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:58.592135 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:12:58.592136 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:12:58.592136 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:12:58.592294 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:12:58.592702 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:12:58.592709 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:12:58.592855 (+ 146us) client_negotiation.cc:769] Sending connection context
0811 05:12:58.592952 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 05:12:58.593107 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":81,"threads_started":1}
I20260811 05:12:58.593748 3136 heartbeater.cc:347] Connected to a master server at 127.28.143.190:45015
I20260811 05:12:58.593848 3136 heartbeater.cc:464] Registering TS with master...
I20260811 05:12:58.594022 3136 heartbeater.cc:511] Master 127.28.143.190:45015 requested a full tablet report, sending...
I20260811 05:12:58.594424 2298 ts_manager.cc:194] Registered new tserver with Master: 6211950da8eb4ec7822999779f957752 (127.28.143.131:33079)
I20260811 05:12:58.594813 2298 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.131@KRBTEST.COM'} at 127.28.143.131:47003
I20260811 05:12:58.600486 29246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 05:12:58 dist-test-slave-f7th krb5kdc[2231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425178, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 05:12:58 dist-test-slave-f7th krb5kdc[2231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425178, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:12:58.626719 2376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:12:58.617639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48428 (local address 127.28.143.190:45015)
0811 05:12:58.617715 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0811 05:12:58.617717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:12:58.617778 (+ 61us) server_negotiation.cc:407] Connection header received
0811 05:12:58.617836 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:12:58.617837 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:12:58.617846 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:12:58.617908 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:12:58.618533 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:58.619055 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:58.619845 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:12:58.620043 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:12:58.620114 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:12:58.623654 (+ 3540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:12:58.623656 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:12:58.623657 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:12:58.623667 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:12:58.625238 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:58.625700 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:58.625701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:58.625702 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:58.625765 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:12:58.626074 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:12:58.626075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:12:58.626075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:12:58.626249 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:12:58.626343 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 05:12:58.626557 (+ 214us) server_negotiation.cc:299] Negotiation successful
0811 05:12:58.626647 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260811 05:12:58.627008 2298 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:48428
I20260811 05:12:58.627511 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 2411
I20260811 05:12:58.648126 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 2658
I20260811 05:12:58.659173 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 2913
I20260811 05:12:58.689627 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 2251
2026-08-11T05:12:58Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2152 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:12:58 dist-test-slave-f7th krb5kdc[3164](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:12:58 dist-test-slave-f7th krb5kdc[3164](info): set up 2 sockets
Aug 11 05:12:58 dist-test-slave-f7th krb5kdc[3164](info): commencing operation
krb5kdc: starting...
W20260811 05:12:59.922825 29246 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Aug 11 05:13:00 dist-test-slave-f7th krb5kdc[3164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:13:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:13:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
I20260811 05:13:00.074981 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:33137
--webserver_interface=127.28.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:42807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:33137
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:00.168923 3194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:00.169162 3194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:00.169185 3194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:00.170387 3194 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:00.170432 3194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:00.170445 3194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:00.170465 3194 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:00.170476 3194 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:13:00.172091 3194 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:42807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:33137
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:33137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:00.172324 3194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:00.172531 3194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:00.177738 3200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:00.177793 3201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:00.177932 3194 server_base.cc:1034] running on GCE node
W20260811 05:13:00.177738 3207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:00.178308 3194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:00.178627 3194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:00.179802 3194 hybrid_clock.cc:648] HybridClock initialized: now 1786425180179790 us; error 32 us; skew 500 ppm
Aug 11 05:13:00 dist-test-slave-f7th krb5kdc[3164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425180, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:00.182768 3194 init.cc:377] Logged in from keytab as kudu/127.28.143.190@KRBTEST.COM (short username kudu)
I20260811 05:13:00.183302 3194 webserver.cc:460] Webserver started at http://127.28.143.190:34991/ using document root <none> and password file <none>
I20260811 05:13:00.183519 3194 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:00.183580 3194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:00.183717 3194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:00.184886 3194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "0e857401af3d447d84a7089ce3878d19"
format_stamp: "Formatted at 2026-08-11 05:13:00 on dist-test-slave-f7th"
server_key: "4ccb0ab78d4052abcc0cfe46cc088ebf"
server_key_iv: "028297184041ec2ce969397327bf12ae"
server_key_version: "encryptionkey@0"
I20260811 05:13:00.185240 3194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "0e857401af3d447d84a7089ce3878d19"
format_stamp: "Formatted at 2026-08-11 05:13:00 on dist-test-slave-f7th"
server_key: "4ccb0ab78d4052abcc0cfe46cc088ebf"
server_key_iv: "028297184041ec2ce969397327bf12ae"
server_key_version: "encryptionkey@0"
I20260811 05:13:00.186628 3194 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 05:13:00.187433 3216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:00.187794 3194 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:00.187919 3194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "0e857401af3d447d84a7089ce3878d19"
format_stamp: "Formatted at 2026-08-11 05:13:00 on dist-test-slave-f7th"
server_key: "4ccb0ab78d4052abcc0cfe46cc088ebf"
server_key_iv: "028297184041ec2ce969397327bf12ae"
server_key_version: "encryptionkey@0"
I20260811 05:13:00.187994 3194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:00.199482 3194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:00.202086 3194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:00.202260 3194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:00.206409 3194 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:33137
I20260811 05:13:00.206446 3315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:33137 every 8 connection(s)
I20260811 05:13:00.206985 3194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:13:00.207736 3316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:00.210250 3316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e857401af3d447d84a7089ce3878d19: Bootstrap starting.
I20260811 05:13:00.210914 3316 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e857401af3d447d84a7089ce3878d19: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:00.211202 3316 log.cc:824] T 00000000000000000000000000000000 P 0e857401af3d447d84a7089ce3878d19: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:00.211580 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 3194
I20260811 05:13:00.211664 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:13:00.211822 29246 external_mini_cluster.cc:1428] Setting key 66e1209da76a7881e626d46ce622a495
I20260811 05:13:00.212036 3316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e857401af3d447d84a7089ce3878d19: No bootstrap required, opened a new log
I20260811 05:13:00.214795 3316 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e857401af3d447d84a7089ce3878d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e857401af3d447d84a7089ce3878d19" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 33137 } }
I20260811 05:13:00.214988 3316 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e857401af3d447d84a7089ce3878d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:00.215025 3316 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e857401af3d447d84a7089ce3878d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e857401af3d447d84a7089ce3878d19, State: Initialized, Role: FOLLOWER
I20260811 05:13:00.215148 3316 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e857401af3d447d84a7089ce3878d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e857401af3d447d84a7089ce3878d19" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 33137 } }
I20260811 05:13:00.215225 3316 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e857401af3d447d84a7089ce3878d19 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:13:00.215265 3316 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e857401af3d447d84a7089ce3878d19 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:13:00.215306 3316 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e857401af3d447d84a7089ce3878d19 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:00.216148 3316 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e857401af3d447d84a7089ce3878d19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e857401af3d447d84a7089ce3878d19" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 33137 } }
I20260811 05:13:00.216280 3316 leader_election.cc:302] T 00000000000000000000000000000000 P 0e857401af3d447d84a7089ce3878d19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e857401af3d447d84a7089ce3878d19; no voters:
I20260811 05:13:00.216459 3316 leader_election.cc:288] T 00000000000000000000000000000000 P 0e857401af3d447d84a7089ce3878d19 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:13:00.216550 3328 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e857401af3d447d84a7089ce3878d19 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:00.216744 3328 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e857401af3d447d84a7089ce3878d19 [term 1 LEADER]: Becoming Leader. State: Replica: 0e857401af3d447d84a7089ce3878d19, State: Running, Role: LEADER
I20260811 05:13:00.216854 3328 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e857401af3d447d84a7089ce3878d19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e857401af3d447d84a7089ce3878d19" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 33137 } }
I20260811 05:13:00.216876 3316 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e857401af3d447d84a7089ce3878d19 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:13:00.217361 3329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e857401af3d447d84a7089ce3878d19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e857401af3d447d84a7089ce3878d19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e857401af3d447d84a7089ce3878d19" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 33137 } } }
I20260811 05:13:00.217446 3329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e857401af3d447d84a7089ce3878d19 [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:00.217633 3331 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e857401af3d447d84a7089ce3878d19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e857401af3d447d84a7089ce3878d19. Latest consensus state: current_term: 1 leader_uuid: "0e857401af3d447d84a7089ce3878d19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e857401af3d447d84a7089ce3878d19" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 33137 } } }
I20260811 05:13:00.217720 3331 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e857401af3d447d84a7089ce3878d19 [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:00.217854 3338 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:13:00.218732 3338 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 11 05:13:00 dist-test-slave-f7th krb5kdc[3164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:13:00.221004 3338 catalog_manager.cc:1347] Generated new cluster ID: c1b20ea30ab140e09b4ecc47debbf174
I20260811 05:13:00.221087 3338 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:13:00.224417 3325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:00.212337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51152 (local address 127.28.143.190:33137)
0811 05:13:00.212554 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:00.212564 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:00.212602 (+ 38us) server_negotiation.cc:407] Connection header received
0811 05:13:00.213420 (+ 818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:00.213430 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:00.213497 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:00.213655 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:00.214310 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:00.215685 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:00.216925 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:00.217543 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:00.217975 (+ 432us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:00.220770 (+ 2795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:00.220775 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:00.220776 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:00.220791 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:00.222599 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:00.222975 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:00.222979 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:00.222979 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:00.223080 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:00.223258 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:00.223259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:00.223260 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:00.223518 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:00.223682 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:00.223792 (+ 110us) server_negotiation.cc:299] Negotiation successful
0811 05:13:00.223936 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":92,"threads_started":1}
I20260811 05:13:00.237902 3338 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:13:00.238489 3338 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:13:00.247112 3338 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0e857401af3d447d84a7089ce3878d19: Generated new TSK 0
I20260811 05:13:00.247391 3338 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.143.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab.
Entry for principal kudu/127.28.143.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab.
I20260811 05:13:00.281421 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.129:0
--local_ip_for_outbound_sockets=127.28.143.129
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:33137
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:42807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:00.370402 3369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:00.370677 3369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:00.370714 3369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:00.371989 3369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:00.372045 3369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:00.372083 3369 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.129
I20260811 05:13:00.373773 3369 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:42807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:33137
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3369
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.129
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:00.374079 3369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:00.374310 3369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:00.379349 3380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:00.379357 3381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:00.379457 3369 server_base.cc:1034] running on GCE node
W20260811 05:13:00.379487 3384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:00.379858 3369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:00.380148 3369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:00.381317 3369 hybrid_clock.cc:648] HybridClock initialized: now 1786425180381302 us; error 28 us; skew 500 ppm
Aug 11 05:13:00 dist-test-slave-f7th krb5kdc[3164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425180, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:00.383961 3369 init.cc:377] Logged in from keytab as kudu/127.28.143.129@KRBTEST.COM (short username kudu)
I20260811 05:13:00.384423 3369 webserver.cc:460] Webserver started at http://127.28.143.129:36729/ using document root <none> and password file <none>
I20260811 05:13:00.384644 3369 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:00.384701 3369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:00.384800 3369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:00.385875 3369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-0/data/instance:
uuid: "05ba9d6f49954fb4bdaabf98472e168e"
format_stamp: "Formatted at 2026-08-11 05:13:00 on dist-test-slave-f7th"
server_key: "95a72deafe2366ae9982d331642f3c89"
server_key_iv: "5f3a3d991a164d449628ea85b858a041"
server_key_version: "encryptionkey@0"
I20260811 05:13:00.386175 3369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance:
uuid: "05ba9d6f49954fb4bdaabf98472e168e"
format_stamp: "Formatted at 2026-08-11 05:13:00 on dist-test-slave-f7th"
server_key: "95a72deafe2366ae9982d331642f3c89"
server_key_iv: "5f3a3d991a164d449628ea85b858a041"
server_key_version: "encryptionkey@0"
I20260811 05:13:00.387552 3369 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 05:13:00.388499 3398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:00.388715 3369 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:00.388823 3369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-0/wal
uuid: "05ba9d6f49954fb4bdaabf98472e168e"
format_stamp: "Formatted at 2026-08-11 05:13:00 on dist-test-slave-f7th"
server_key: "95a72deafe2366ae9982d331642f3c89"
server_key_iv: "5f3a3d991a164d449628ea85b858a041"
server_key_version: "encryptionkey@0"
I20260811 05:13:00.388904 3369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:00.407723 3369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:00.409863 3369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:00.409996 3369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:00.410228 3369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:00.410506 3369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:00.410537 3369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:00.410560 3369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:00.410574 3369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:00.416170 3369 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.129:41593
I20260811 05:13:00.416303 3599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.129:41593 every 8 connection(s)
I20260811 05:13:00.416584 3369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
I20260811 05:13:00.418569 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 3369
I20260811 05:13:00.418665 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance
I20260811 05:13:00.418866 29246 external_mini_cluster.cc:1428] Setting key bf8d07c0d4094c84b3a8f91b4e0516a3
Aug 11 05:13:00 dist-test-slave-f7th krb5kdc[3164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425180, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:13:00.426467 3325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:00.417369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:58769 (local address 127.28.143.190:33137)
0811 05:13:00.417425 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:00.417428 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:00.417667 (+ 239us) server_negotiation.cc:407] Connection header received
0811 05:13:00.418679 (+ 1012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:00.418681 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:00.418694 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:00.418761 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:00.419494 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:00.419998 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:00.421150 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:00.421361 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:00.421418 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:00.423717 (+ 2299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:00.423719 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:00.423720 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:00.423729 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:00.425097 (+ 1368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:00.425519 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:00.425520 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:00.425521 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:00.425578 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:00.425814 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:00.425815 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:00.425815 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:00.425936 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:00.425993 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:00.426275 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 05:13:00.426370 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 05:13:00.426656 3608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:00.417344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:33137 (local address 127.28.143.129:58769)
0811 05:13:00.417513 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:00.417553 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:00.418572 (+ 1019us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:00.418818 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:00.418823 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:00.419203 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:00.419400 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:00.419404 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:00.420084 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:00.420085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:00.421062 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:00.421066 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:00.421436 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:00.421437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:00.421563 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:00.422068 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:00.422077 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:00.423610 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:00.425183 (+ 1573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:00.425194 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:00.425196 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:00.425442 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:00.425634 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:00.425635 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:00.425635 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:00.425755 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:00.425999 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:00.426007 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:00.426153 (+ 146us) client_negotiation.cc:769] Sending connection context
0811 05:13:00.426246 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 05:13:00.426418 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":93,"threads_started":1}
WARNING: no policy specified for kudu/127.28.143.130@KRBTEST.COM; defaulting to no policy
I20260811 05:13:00.427408 3601 heartbeater.cc:347] Connected to a master server at 127.28.143.190:33137
I20260811 05:13:00.427518 3601 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:00.427734 3601 heartbeater.cc:511] Master 127.28.143.190:33137 requested a full tablet report, sending...
I20260811 05:13:00.428364 3244 ts_manager.cc:194] Registered new tserver with Master: 05ba9d6f49954fb4bdaabf98472e168e (127.28.143.129:41593)
I20260811 05:13:00.429096 3244 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.129@KRBTEST.COM'} at 127.28.143.129:58769
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab.
Entry for principal kudu/127.28.143.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab.
I20260811 05:13:00.448712 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.130:0
--local_ip_for_outbound_sockets=127.28.143.130
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:33137
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:42807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:00.540117 3611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:00.540349 3611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:00.540373 3611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:00.541625 3611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:00.541679 3611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:00.541708 3611 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.130
I20260811 05:13:00.543298 3611 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:42807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:33137
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.130
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:00.543526 3611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:00.543720 3611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:00.548980 3622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:00.548985 3618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:00.548974 3617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:00.549208 3611 server_base.cc:1034] running on GCE node
I20260811 05:13:00.549547 3611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:00.549791 3611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:00.550962 3611 hybrid_clock.cc:648] HybridClock initialized: now 1786425180550942 us; error 29 us; skew 500 ppm
Aug 11 05:13:00 dist-test-slave-f7th krb5kdc[3164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425180, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:00.553799 3611 init.cc:377] Logged in from keytab as kudu/127.28.143.130@KRBTEST.COM (short username kudu)
I20260811 05:13:00.554330 3611 webserver.cc:460] Webserver started at http://127.28.143.130:45905/ using document root <none> and password file <none>
I20260811 05:13:00.554560 3611 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:00.554603 3611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:00.554695 3611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:00.555745 3611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-1/data/instance:
uuid: "dd92b04eca43406b9fac82985426dc2b"
format_stamp: "Formatted at 2026-08-11 05:13:00 on dist-test-slave-f7th"
server_key: "e824e403e27816207d0738541b8e0b36"
server_key_iv: "edaa007d8561175147fa7729d738f3ef"
server_key_version: "encryptionkey@0"
I20260811 05:13:00.556063 3611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance:
uuid: "dd92b04eca43406b9fac82985426dc2b"
format_stamp: "Formatted at 2026-08-11 05:13:00 on dist-test-slave-f7th"
server_key: "e824e403e27816207d0738541b8e0b36"
server_key_iv: "edaa007d8561175147fa7729d738f3ef"
server_key_version: "encryptionkey@0"
I20260811 05:13:00.557477 3611 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 05:13:00.558280 3644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:00.558588 3611 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 05:13:00.558696 3611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-1/wal
uuid: "dd92b04eca43406b9fac82985426dc2b"
format_stamp: "Formatted at 2026-08-11 05:13:00 on dist-test-slave-f7th"
server_key: "e824e403e27816207d0738541b8e0b36"
server_key_iv: "edaa007d8561175147fa7729d738f3ef"
server_key_version: "encryptionkey@0"
I20260811 05:13:00.558768 3611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:00.571383 3611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:00.573701 3611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:00.573856 3611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:00.574131 3611 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:00.574471 3611 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:00.574512 3611 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:00.574544 3611 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:00.574565 3611 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:00.580816 3611 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.130:44123
I20260811 05:13:00.580909 3842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.130:44123 every 8 connection(s)
I20260811 05:13:00.581229 3611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
I20260811 05:13:00.585443 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 3611
I20260811 05:13:00.585538 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance
I20260811 05:13:00.585711 29246 external_mini_cluster.cc:1428] Setting key c20ece29c8523c0a572d127e31a4211c
Aug 11 05:13:00 dist-test-slave-f7th krb5kdc[3164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425180, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:13:00.591073 3325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:00.582067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:36429 (local address 127.28.143.190:33137)
0811 05:13:00.582136 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:00.582139 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:00.582321 (+ 182us) server_negotiation.cc:407] Connection header received
0811 05:13:00.583323 (+ 1002us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:00.583324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:00.583338 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:00.583408 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:00.584114 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:00.584641 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:00.585707 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:00.585927 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:00.585981 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:00.588202 (+ 2221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:00.588205 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:00.588205 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:00.588213 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:00.589584 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:00.589986 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:00.589988 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:00.589988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:00.590066 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:00.590295 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:00.590295 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:00.590296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:00.590455 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:00.590522 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:00.590865 (+ 343us) server_negotiation.cc:299] Negotiation successful
0811 05:13:00.590960 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 05:13:00.591264 3850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:00.582004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:33137 (local address 127.28.143.130:36429)
0811 05:13:00.582202 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:00.582243 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:00.583208 (+ 965us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:00.583465 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:00.583470 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:00.583766 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:00.584004 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:00.584009 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:00.584740 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:00.584741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:00.585627 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:00.585630 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:00.586011 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:00.586013 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:00.586171 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:00.586712 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:00.586721 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:00.588110 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:00.589661 (+ 1551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:00.589668 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:00.589670 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:00.589907 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:00.590129 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:00.590130 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:00.590130 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:00.590239 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:00.590550 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:00.590559 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:00.590713 (+ 154us) client_negotiation.cc:769] Sending connection context
0811 05:13:00.590818 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 05:13:00.590998 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":85,"threads_started":1}
I20260811 05:13:00.591917 3848 heartbeater.cc:347] Connected to a master server at 127.28.143.190:33137
I20260811 05:13:00.592061 3848 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:00.592290 3848 heartbeater.cc:511] Master 127.28.143.190:33137 requested a full tablet report, sending...
I20260811 05:13:00.592788 3244 ts_manager.cc:194] Registered new tserver with Master: dd92b04eca43406b9fac82985426dc2b (127.28.143.130:44123)
I20260811 05:13:00.593341 3244 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.130@KRBTEST.COM'} at 127.28.143.130:36429
WARNING: no policy specified for kudu/127.28.143.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab.
Entry for principal kudu/127.28.143.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab.
I20260811 05:13:00.619971 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.131:0
--local_ip_for_outbound_sockets=127.28.143.131
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:33137
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:42807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:00.709647 3856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:00.709918 3856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:00.709955 3856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:00.711314 3856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:00.711374 3856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:00.711411 3856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.131
I20260811 05:13:00.713212 3856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:42807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:33137
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.131
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:00.713492 3856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:00.713740 3856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:00.719439 3869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:00.719444 3876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:00.719444 3867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:00.719795 3856 server_base.cc:1034] running on GCE node
I20260811 05:13:00.720043 3856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:00.720290 3856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:00.721462 3856 hybrid_clock.cc:648] HybridClock initialized: now 1786425180721448 us; error 34 us; skew 500 ppm
Aug 11 05:13:00 dist-test-slave-f7th krb5kdc[3164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425180, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:00.724608 3856 init.cc:377] Logged in from keytab as kudu/127.28.143.131@KRBTEST.COM (short username kudu)
I20260811 05:13:00.725167 3856 webserver.cc:460] Webserver started at http://127.28.143.131:42547/ using document root <none> and password file <none>
I20260811 05:13:00.725390 3856 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:00.725445 3856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:00.725577 3856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:00.726717 3856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-2/data/instance:
uuid: "481f95e1846d49d3a0d798cd97028abf"
format_stamp: "Formatted at 2026-08-11 05:13:00 on dist-test-slave-f7th"
server_key: "73b9c17b69bdabeef4a9511786afdad9"
server_key_iv: "4bad1058094ef865e3891478f84c7c76"
server_key_version: "encryptionkey@0"
I20260811 05:13:00.727097 3856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance:
uuid: "481f95e1846d49d3a0d798cd97028abf"
format_stamp: "Formatted at 2026-08-11 05:13:00 on dist-test-slave-f7th"
server_key: "73b9c17b69bdabeef4a9511786afdad9"
server_key_iv: "4bad1058094ef865e3891478f84c7c76"
server_key_version: "encryptionkey@0"
I20260811 05:13:00.728938 3856 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:13:00.729852 3893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:00.730042 3856 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:00.730145 3856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-2/wal
uuid: "481f95e1846d49d3a0d798cd97028abf"
format_stamp: "Formatted at 2026-08-11 05:13:00 on dist-test-slave-f7th"
server_key: "73b9c17b69bdabeef4a9511786afdad9"
server_key_iv: "4bad1058094ef865e3891478f84c7c76"
server_key_version: "encryptionkey@0"
I20260811 05:13:00.730216 3856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:00.754560 3856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:00.756984 3856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:00.757148 3856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:00.757419 3856 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:00.757754 3856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:00.757791 3856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:00.757822 3856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:00.757846 3856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:00.764690 3856 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.131:44059
I20260811 05:13:00.764781 4110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.131:44059 every 8 connection(s)
I20260811 05:13:00.765122 3856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
I20260811 05:13:00.767163 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 3856
I20260811 05:13:00.767261 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance
I20260811 05:13:00.767472 29246 external_mini_cluster.cc:1428] Setting key 5993eb51439781c4de837b3dac85f0f3
Aug 11 05:13:00 dist-test-slave-f7th krb5kdc[3164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425180, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:13:00.776278 3325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:00.766001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:38365 (local address 127.28.143.190:33137)
0811 05:13:00.766133 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:00.766136 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:00.766313 (+ 177us) server_negotiation.cc:407] Connection header received
0811 05:13:00.767333 (+ 1020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:00.767335 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:00.767349 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:00.767405 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:00.768360 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:00.769060 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:00.770165 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:00.770365 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:00.770568 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:00.773106 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:00.773108 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:00.773108 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:00.773117 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:00.774587 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:00.775112 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:00.775113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:00.775114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:00.775184 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:00.775501 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:00.775502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:00.775502 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:00.775652 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:00.775712 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:00.776116 (+ 404us) server_negotiation.cc:299] Negotiation successful
0811 05:13:00.776200 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 05:13:00.776526 4115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:00.765913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:33137 (local address 127.28.143.131:38365)
0811 05:13:00.766157 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:00.766219 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:00.767199 (+ 980us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:00.767480 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:00.767484 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:00.767869 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:00.768247 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:00.768256 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:00.769156 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:00.769157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:00.770046 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:00.770049 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:00.770577 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:00.770578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:00.770699 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:00.771251 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:00.771261 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:00.772970 (+ 1709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:00.774690 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:00.774699 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:00.774701 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:00.775014 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:00.775283 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:00.775284 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:00.775285 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:00.775427 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:00.775754 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:00.775763 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:00.775962 (+ 199us) client_negotiation.cc:769] Sending connection context
0811 05:13:00.776068 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 05:13:00.776235 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":86,"threads_started":1}
I20260811 05:13:00.777153 4112 heartbeater.cc:347] Connected to a master server at 127.28.143.190:33137
I20260811 05:13:00.777258 4112 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:00.777436 4112 heartbeater.cc:511] Master 127.28.143.190:33137 requested a full tablet report, sending...
I20260811 05:13:00.777879 3244 ts_manager.cc:194] Registered new tserver with Master: 481f95e1846d49d3a0d798cd97028abf (127.28.143.131:44059)
I20260811 05:13:00.778264 3244 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.131@KRBTEST.COM'} at 127.28.143.131:38365
I20260811 05:13:00.779748 29246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 05:13:00 dist-test-slave-f7th krb5kdc[3164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 05:13:00 dist-test-slave-f7th krb5kdc[3164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.143.129@KRBTEST.COM
I20260811 05:13:00.806885 4122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:00.798166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53652 (local address 127.28.143.129:41593)
0811 05:13:00.798333 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:00.798340 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:00.798355 (+ 15us) server_negotiation.cc:407] Connection header received
0811 05:13:00.798388 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:00.798391 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:00.798405 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:00.798491 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:00.799183 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:00.799929 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:00.800759 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:00.801118 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:00.801180 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:00.803645 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:00.803659 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:00.803659 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:00.803668 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:00.805330 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:00.805758 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:00.805762 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:00.805763 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:00.805852 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:00.806202 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:00.806204 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:00.806204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:00.806374 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:00.806507 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:00.806657 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 05:13:00.806740 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":77,"threads_started":1}
Aug 11 05:13:00 dist-test-slave-f7th krb5kdc[3164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:13:00.816105 3325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:00.808382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33416 (local address 127.28.143.190:33137)
0811 05:13:00.808454 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:00.808456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:00.808465 (+ 9us) server_negotiation.cc:407] Connection header received
0811 05:13:00.808495 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:00.808495 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:00.808505 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:00.808552 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:00.809074 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:00.809580 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:00.810313 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:00.810507 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:00.810567 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:00.813010 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:00.813012 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:00.813012 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:00.813023 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:00.814634 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:00.815101 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:00.815103 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:00.815104 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:00.815173 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:00.815459 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:00.815460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:00.815461 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:00.815646 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:00.815746 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:00.815869 (+ 123us) server_negotiation.cc:299] Negotiation successful
0811 05:13:00.815994 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260811 05:13:00.816437 3244 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33416
I20260811 05:13:00.816872 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 3369
I20260811 05:13:00.829226 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 3611
I20260811 05:13:00.847033 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 3856
I20260811 05:13:00.871181 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 3194
2026-08-11T05:13:00Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2187 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:13:00 dist-test-slave-f7th krb5kdc[4133](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:13:00 dist-test-slave-f7th krb5kdc[4133](info): set up 2 sockets
Aug 11 05:13:00 dist-test-slave-f7th krb5kdc[4133](info): commencing operation
krb5kdc: starting...
W20260811 05:13:02.102555 29246 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Aug 11 05:13:02 dist-test-slave-f7th krb5kdc[4133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:13:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:13:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
I20260811 05:13:02.237637 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:32805
--webserver_interface=127.28.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:35297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:32805
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:02.324786 4151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:02.325030 4151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:02.325066 4151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:02.326547 4151 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:02.326632 4151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:02.326654 4151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:02.326674 4151 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:02.326690 4151 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:13:02.328660 4151 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:35297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:32805
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:32805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4151
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:02.328990 4151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:02.329245 4151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:02.335008 4162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:02.335080 4164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:02.335022 4166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:02.335248 4151 server_base.cc:1034] running on GCE node
I20260811 05:13:02.335641 4151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:02.335970 4151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:02.337153 4151 hybrid_clock.cc:648] HybridClock initialized: now 1786425182337130 us; error 34 us; skew 500 ppm
Aug 11 05:13:02 dist-test-slave-f7th krb5kdc[4133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425182, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:02.343995 4151 init.cc:377] Logged in from keytab as kudu/127.28.143.190@KRBTEST.COM (short username kudu)
I20260811 05:13:02.344354 4151 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:02.344404 4151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:02.344506 4151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:02.345685 4151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "218cebc95c51478fb9dc0ba1b1f40fde"
format_stamp: "Formatted at 2026-08-11 05:13:02 on dist-test-slave-f7th"
server_key: "96602d8603056b02132267acb053cce9"
server_key_iv: "e058cdafe91a2b5799eaf0c454f7baaa"
server_key_version: "encryptionkey@0"
I20260811 05:13:02.346012 4151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "218cebc95c51478fb9dc0ba1b1f40fde"
format_stamp: "Formatted at 2026-08-11 05:13:02 on dist-test-slave-f7th"
server_key: "96602d8603056b02132267acb053cce9"
server_key_iv: "e058cdafe91a2b5799eaf0c454f7baaa"
server_key_version: "encryptionkey@0"
I20260811 05:13:02.347294 4151 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 05:13:02.348111 4177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:02.348320 4151 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:02.348425 4151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "218cebc95c51478fb9dc0ba1b1f40fde"
format_stamp: "Formatted at 2026-08-11 05:13:02 on dist-test-slave-f7th"
server_key: "96602d8603056b02132267acb053cce9"
server_key_iv: "e058cdafe91a2b5799eaf0c454f7baaa"
server_key_version: "encryptionkey@0"
I20260811 05:13:02.348492 4151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:02.360450 4151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:02.362720 4151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:02.362857 4151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:02.367131 4151 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:32805
I20260811 05:13:02.367177 4270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:32805 every 8 connection(s)
I20260811 05:13:02.367475 4151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:13:02.368683 4272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:02.371122 4272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 218cebc95c51478fb9dc0ba1b1f40fde: Bootstrap starting.
I20260811 05:13:02.371802 4272 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 218cebc95c51478fb9dc0ba1b1f40fde: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:02.372094 4272 log.cc:824] T 00000000000000000000000000000000 P 218cebc95c51478fb9dc0ba1b1f40fde: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:02.372881 4272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 218cebc95c51478fb9dc0ba1b1f40fde: No bootstrap required, opened a new log
I20260811 05:13:02.374480 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 4151
I20260811 05:13:02.374595 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:13:02.374786 29246 external_mini_cluster.cc:1428] Setting key bc4a07ac292f412839084d869a79e6c3
I20260811 05:13:02.374688 4272 raft_consensus.cc:348] T 00000000000000000000000000000000 P 218cebc95c51478fb9dc0ba1b1f40fde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "218cebc95c51478fb9dc0ba1b1f40fde" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 32805 } }
I20260811 05:13:02.374887 4272 raft_consensus.cc:374] T 00000000000000000000000000000000 P 218cebc95c51478fb9dc0ba1b1f40fde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:02.374933 4272 raft_consensus.cc:729] T 00000000000000000000000000000000 P 218cebc95c51478fb9dc0ba1b1f40fde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 218cebc95c51478fb9dc0ba1b1f40fde, State: Initialized, Role: FOLLOWER
I20260811 05:13:02.375083 4272 consensus_queue.cc:260] T 00000000000000000000000000000000 P 218cebc95c51478fb9dc0ba1b1f40fde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "218cebc95c51478fb9dc0ba1b1f40fde" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 32805 } }
I20260811 05:13:02.375160 4272 raft_consensus.cc:388] T 00000000000000000000000000000000 P 218cebc95c51478fb9dc0ba1b1f40fde [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:13:02.375183 4272 raft_consensus.cc:482] T 00000000000000000000000000000000 P 218cebc95c51478fb9dc0ba1b1f40fde [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:13:02.375211 4272 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 218cebc95c51478fb9dc0ba1b1f40fde [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:02.376338 4272 raft_consensus.cc:504] T 00000000000000000000000000000000 P 218cebc95c51478fb9dc0ba1b1f40fde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "218cebc95c51478fb9dc0ba1b1f40fde" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 32805 } }
I20260811 05:13:02.376446 4272 leader_election.cc:302] T 00000000000000000000000000000000 P 218cebc95c51478fb9dc0ba1b1f40fde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 218cebc95c51478fb9dc0ba1b1f40fde; no voters:
I20260811 05:13:02.376607 4272 leader_election.cc:288] T 00000000000000000000000000000000 P 218cebc95c51478fb9dc0ba1b1f40fde [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:13:02.376668 4282 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 218cebc95c51478fb9dc0ba1b1f40fde [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:02.376822 4282 raft_consensus.cc:686] T 00000000000000000000000000000000 P 218cebc95c51478fb9dc0ba1b1f40fde [term 1 LEADER]: Becoming Leader. State: Replica: 218cebc95c51478fb9dc0ba1b1f40fde, State: Running, Role: LEADER
I20260811 05:13:02.376919 4272 sys_catalog.cc:564] T 00000000000000000000000000000000 P 218cebc95c51478fb9dc0ba1b1f40fde [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:13:02.376982 4282 consensus_queue.cc:237] T 00000000000000000000000000000000 P 218cebc95c51478fb9dc0ba1b1f40fde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "218cebc95c51478fb9dc0ba1b1f40fde" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 32805 } }
I20260811 05:13:02.377403 4282 sys_catalog.cc:455] T 00000000000000000000000000000000 P 218cebc95c51478fb9dc0ba1b1f40fde [sys.catalog]: SysCatalogTable state changed. Reason: New leader 218cebc95c51478fb9dc0ba1b1f40fde. Latest consensus state: current_term: 1 leader_uuid: "218cebc95c51478fb9dc0ba1b1f40fde" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "218cebc95c51478fb9dc0ba1b1f40fde" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 32805 } } }
I20260811 05:13:02.377669 4282 sys_catalog.cc:458] T 00000000000000000000000000000000 P 218cebc95c51478fb9dc0ba1b1f40fde [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:02.377588 4286 sys_catalog.cc:455] T 00000000000000000000000000000000 P 218cebc95c51478fb9dc0ba1b1f40fde [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "218cebc95c51478fb9dc0ba1b1f40fde" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "218cebc95c51478fb9dc0ba1b1f40fde" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 32805 } } }
I20260811 05:13:02.377995 4286 sys_catalog.cc:458] T 00000000000000000000000000000000 P 218cebc95c51478fb9dc0ba1b1f40fde [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:02.378546 4306 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:13:02.379097 4306 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:13:02.381214 4306 catalog_manager.cc:1347] Generated new cluster ID: b7d132e5914041219cf8277bea9321e4
I20260811 05:13:02.381278 4306 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 05:13:02 dist-test-slave-f7th krb5kdc[4133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:13:02.386376 4281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:02.375307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38026 (local address 127.28.143.190:32805)
0811 05:13:02.375517 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:02.375527 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:02.375560 (+ 33us) server_negotiation.cc:407] Connection header received
0811 05:13:02.375700 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:02.375710 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:02.375783 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:02.375972 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:02.376647 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:02.378505 (+ 1858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:02.379395 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:02.379908 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:02.380012 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:02.382753 (+ 2741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:02.382759 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:02.382759 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:02.382774 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:02.384415 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:02.384828 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:02.384832 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:02.384832 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:02.385266 (+ 434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:02.385476 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:02.385477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:02.385477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:02.385748 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:02.385903 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:02.386019 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 05:13:02.386170 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":101,"threads_started":1}
I20260811 05:13:02.420392 4306 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:13:02.421284 4306 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:13:02.428794 4306 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 218cebc95c51478fb9dc0ba1b1f40fde: Generated new TSK 0
I20260811 05:13:02.429059 4306 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.143.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab.
Entry for principal kudu/127.28.143.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab.
W20260811 05:13:02.843115 29246 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.28.143.129: real 0.403s user 0.000s sys 0.004s
I20260811 05:13:02.843489 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.129:0
--local_ip_for_outbound_sockets=127.28.143.129
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:32805
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:35297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:02.940292 4320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:02.940534 4320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:02.940564 4320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:02.942162 4320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:02.942243 4320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:02.942276 4320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.129
I20260811 05:13:02.944780 4320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:35297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:32805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.129
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:02.945086 4320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:02.945333 4320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:02.951050 4336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:02.950994 4328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:02.951004 4329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:02.951270 4320 server_base.cc:1034] running on GCE node
I20260811 05:13:02.951591 4320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:02.951884 4320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:02.953043 4320 hybrid_clock.cc:648] HybridClock initialized: now 1786425182953021 us; error 33 us; skew 500 ppm
Aug 11 05:13:02 dist-test-slave-f7th krb5kdc[4133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425182, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:02.955746 4320 init.cc:377] Logged in from keytab as kudu/127.28.143.129@KRBTEST.COM (short username kudu)
I20260811 05:13:02.956128 4320 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:02.956180 4320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:02.956305 4320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:02.957484 4320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-0/data/instance:
uuid: "89a0a1577ee94a168f79c3edcebe0f08"
format_stamp: "Formatted at 2026-08-11 05:13:02 on dist-test-slave-f7th"
server_key: "5999acd5e6ff76b0ff5b1105f521ab40"
server_key_iv: "d3394019a6344ed94864550cdeebf2a1"
server_key_version: "encryptionkey@0"
I20260811 05:13:02.957836 4320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance:
uuid: "89a0a1577ee94a168f79c3edcebe0f08"
format_stamp: "Formatted at 2026-08-11 05:13:02 on dist-test-slave-f7th"
server_key: "5999acd5e6ff76b0ff5b1105f521ab40"
server_key_iv: "d3394019a6344ed94864550cdeebf2a1"
server_key_version: "encryptionkey@0"
I20260811 05:13:02.959290 4320 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 05:13:02.960196 4354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:02.960410 4320 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:02.960510 4320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-0/wal
uuid: "89a0a1577ee94a168f79c3edcebe0f08"
format_stamp: "Formatted at 2026-08-11 05:13:02 on dist-test-slave-f7th"
server_key: "5999acd5e6ff76b0ff5b1105f521ab40"
server_key_iv: "d3394019a6344ed94864550cdeebf2a1"
server_key_version: "encryptionkey@0"
I20260811 05:13:02.960577 4320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:02.991889 4320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:02.994578 4320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:02.994726 4320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:02.994997 4320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:02.995366 4320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:02.995405 4320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:02.995437 4320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:02.995448 4320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:03.001428 4320 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.129:35831
I20260811 05:13:03.001482 4567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.129:35831 every 8 connection(s)
I20260811 05:13:03.001763 4320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
Aug 11 05:13:03 dist-test-slave-f7th krb5kdc[4133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425182, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:13:03.012092 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 4320
I20260811 05:13:03.012189 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance
I20260811 05:13:03.012369 29246 external_mini_cluster.cc:1428] Setting key 73b386ffccd55c9ad5713b2fdf0b816a
I20260811 05:13:03.014101 4574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:03.002543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:51993 (local address 127.28.143.190:32805)
0811 05:13:03.003502 (+ 959us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:03.003504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:03.003511 (+ 7us) server_negotiation.cc:407] Connection header received
0811 05:13:03.003536 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:03.003537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:03.003551 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:03.003611 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:03.004974 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:03.005555 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:03.006724 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:03.007099 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:03.007168 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:03.010390 (+ 3222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:03.010392 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:03.010392 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:03.010402 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:03.012360 (+ 1958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:03.012962 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:03.012964 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:03.012965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:03.013054 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:03.013335 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:03.013336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:03.013337 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:03.013485 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:03.013550 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:03.013872 (+ 322us) server_negotiation.cc:299] Negotiation successful
0811 05:13:03.013979 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":936,"thread_start_us":69,"threads_started":1}
I20260811 05:13:03.014266 4573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:03.002500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:32805 (local address 127.28.143.129:51993)
0811 05:13:03.002737 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:03.002783 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:03.003346 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:03.003805 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:03.003809 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:03.004126 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:03.004776 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:03.004783 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:03.005696 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:03.005697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:03.006595 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:03.006599 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:03.007260 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:03.007261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:03.007425 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:03.008219 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:03.008232 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:03.010255 (+ 2023us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:03.012498 (+ 2243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:03.012514 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:03.012517 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:03.012872 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:03.013122 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:03.013122 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:03.013123 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:03.013256 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:03.013553 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:03.013562 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:03.013716 (+ 154us) client_negotiation.cc:769] Sending connection context
0811 05:13:03.013816 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 05:13:03.014018 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":97,"threads_started":1}
I20260811 05:13:03.014894 4570 heartbeater.cc:347] Connected to a master server at 127.28.143.190:32805
I20260811 05:13:03.015038 4570 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:03.015211 4570 heartbeater.cc:511] Master 127.28.143.190:32805 requested a full tablet report, sending...
I20260811 05:13:03.015913 4207 ts_manager.cc:194] Registered new tserver with Master: 89a0a1577ee94a168f79c3edcebe0f08 (127.28.143.129:35831)
I20260811 05:13:03.016770 4207 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.129@KRBTEST.COM'} at 127.28.143.129:51993
WARNING: no policy specified for kudu/127.28.143.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.130@KRBTEST.COM" created.
I20260811 05:13:04.018014 4570 heartbeater.cc:503] Master 127.28.143.190:32805 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab.
Entry for principal kudu/127.28.143.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab.
W20260811 05:13:04.325588 29246 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.28.143.130: real 1.312s user 0.002s sys 0.002s
I20260811 05:13:04.326021 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.130:0
--local_ip_for_outbound_sockets=127.28.143.130
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:32805
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:35297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:04.413961 4582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:04.414211 4582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:04.414244 4582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:04.415617 4582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:04.415683 4582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:04.415715 4582 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.130
I20260811 05:13:04.417529 4582 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:35297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:32805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4582
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.130
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:04.417855 4582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:04.418113 4582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:04.423624 4591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:04.423729 4598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:04.423624 4592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:04.423624 4582 server_base.cc:1034] running on GCE node
I20260811 05:13:04.424250 4582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:04.424522 4582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:04.425685 4582 hybrid_clock.cc:648] HybridClock initialized: now 1786425184425675 us; error 47 us; skew 500 ppm
Aug 11 05:13:04 dist-test-slave-f7th krb5kdc[4133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425184, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:04.428516 4582 init.cc:377] Logged in from keytab as kudu/127.28.143.130@KRBTEST.COM (short username kudu)
I20260811 05:13:04.428929 4582 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:04.428982 4582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:04.429112 4582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:04.430377 4582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-1/data/instance:
uuid: "39fd45c64c71497b8226e52caaf3dbde"
format_stamp: "Formatted at 2026-08-11 05:13:04 on dist-test-slave-f7th"
server_key: "1ab9a1aa465255d36dc75da95a135416"
server_key_iv: "8f2c4ad4831021317cbc2730e7bd49e1"
server_key_version: "encryptionkey@0"
I20260811 05:13:04.431089 4582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance:
uuid: "39fd45c64c71497b8226e52caaf3dbde"
format_stamp: "Formatted at 2026-08-11 05:13:04 on dist-test-slave-f7th"
server_key: "1ab9a1aa465255d36dc75da95a135416"
server_key_iv: "8f2c4ad4831021317cbc2730e7bd49e1"
server_key_version: "encryptionkey@0"
I20260811 05:13:04.432562 4582 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260811 05:13:04.433482 4609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:04.433707 4582 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 05:13:04.433825 4582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-1/wal
uuid: "39fd45c64c71497b8226e52caaf3dbde"
format_stamp: "Formatted at 2026-08-11 05:13:04 on dist-test-slave-f7th"
server_key: "1ab9a1aa465255d36dc75da95a135416"
server_key_iv: "8f2c4ad4831021317cbc2730e7bd49e1"
server_key_version: "encryptionkey@0"
I20260811 05:13:04.433895 4582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:04.469003 4582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:04.471760 4582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:04.471936 4582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:04.472249 4582 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:04.473120 4582 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:04.473176 4582 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:04.473212 4582 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:04.473228 4582 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:04.479849 4582 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.130:35843
I20260811 05:13:04.479894 4822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.130:35843 every 8 connection(s)
I20260811 05:13:04.480250 4582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
I20260811 05:13:04.482962 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 4582
I20260811 05:13:04.483134 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance
I20260811 05:13:04.483407 29246 external_mini_cluster.cc:1428] Setting key 30938b806c787ff947ed778370397e3c
Aug 11 05:13:04 dist-test-slave-f7th krb5kdc[4133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425184, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:13:04.492507 4827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.481308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:60643 (local address 127.28.143.190:32805)
0811 05:13:04.481486 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:04.481488 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:04.481656 (+ 168us) server_negotiation.cc:407] Connection header received
0811 05:13:04.482175 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:04.482176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:04.482190 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:04.482266 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:04.483350 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.484050 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.485652 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.485923 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.485991 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:04.488912 (+ 2921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:04.488913 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:04.488913 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:04.488923 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:04.490581 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:04.491222 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:04.491223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:04.491224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:04.491301 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:04.491674 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:04.491675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:04.491676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:04.491825 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:04.491959 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:04.492318 (+ 359us) server_negotiation.cc:299] Negotiation successful
0811 05:13:04.492416 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":80,"threads_started":1}
I20260811 05:13:04.492762 4828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.481265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:32805 (local address 127.28.143.130:60643)
0811 05:13:04.481498 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:04.481545 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:04.482053 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:04.482326 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:04.482329 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:04.482987 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:04.483246 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:04.483253 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.484160 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.484162 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:04.485471 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:04.485476 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.486012 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.486013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:04.486153 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:04.486786 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:04.486797 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:04.488756 (+ 1959us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:04.490691 (+ 1935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:04.490704 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:04.490707 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:04.491090 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:04.491381 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:04.491382 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:04.491382 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:04.491563 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:04.491905 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:04.491914 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:04.492113 (+ 199us) client_negotiation.cc:769] Sending connection context
0811 05:13:04.492262 (+ 149us) client_negotiation.cc:240] Negotiation successful
0811 05:13:04.492466 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":146,"threads_started":1}
I20260811 05:13:04.493352 4825 heartbeater.cc:347] Connected to a master server at 127.28.143.190:32805
I20260811 05:13:04.493477 4825 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:04.493649 4825 heartbeater.cc:511] Master 127.28.143.190:32805 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.28.143.131@KRBTEST.COM; defaulting to no policy
I20260811 05:13:04.494136 4207 ts_manager.cc:194] Registered new tserver with Master: 39fd45c64c71497b8226e52caaf3dbde (127.28.143.130:35843)
I20260811 05:13:04.494539 4207 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.130@KRBTEST.COM'} at 127.28.143.130:60643
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab.
Entry for principal kudu/127.28.143.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab.
I20260811 05:13:04.520725 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.131:0
--local_ip_for_outbound_sockets=127.28.143.131
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:32805
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:35297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:04.607491 4833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:04.607712 4833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:04.607748 4833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:04.608960 4833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:04.609002 4833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:04.609038 4833 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.131
I20260811 05:13:04.610602 4833 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:35297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:32805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4833
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.131
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:04.610857 4833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:04.611095 4833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:04.616312 4846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:04.616305 4848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:04.616305 4841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:04.616549 4833 server_base.cc:1034] running on GCE node
I20260811 05:13:04.616861 4833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:04.617125 4833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:04.618294 4833 hybrid_clock.cc:648] HybridClock initialized: now 1786425184618233 us; error 79 us; skew 500 ppm
Aug 11 05:13:04 dist-test-slave-f7th krb5kdc[4133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425184, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:04.621187 4833 init.cc:377] Logged in from keytab as kudu/127.28.143.131@KRBTEST.COM (short username kudu)
I20260811 05:13:04.621559 4833 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:04.621614 4833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:04.621748 4833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:04.622864 4833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-2/data/instance:
uuid: "0bbaa1b956294d61a8d03a64257d5506"
format_stamp: "Formatted at 2026-08-11 05:13:04 on dist-test-slave-f7th"
server_key: "72ef819e1ef604269c7af633b5a1f354"
server_key_iv: "cdf9ed0c0bdf65617f91f6f23e4919a9"
server_key_version: "encryptionkey@0"
I20260811 05:13:04.623221 4833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance:
uuid: "0bbaa1b956294d61a8d03a64257d5506"
format_stamp: "Formatted at 2026-08-11 05:13:04 on dist-test-slave-f7th"
server_key: "72ef819e1ef604269c7af633b5a1f354"
server_key_iv: "cdf9ed0c0bdf65617f91f6f23e4919a9"
server_key_version: "encryptionkey@0"
I20260811 05:13:04.624606 4833 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 05:13:04.625626 4862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:04.625924 4833 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:04.626029 4833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-2/wal
uuid: "0bbaa1b956294d61a8d03a64257d5506"
format_stamp: "Formatted at 2026-08-11 05:13:04 on dist-test-slave-f7th"
server_key: "72ef819e1ef604269c7af633b5a1f354"
server_key_iv: "cdf9ed0c0bdf65617f91f6f23e4919a9"
server_key_version: "encryptionkey@0"
I20260811 05:13:04.626113 4833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:04.637054 4833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:04.639393 4833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:04.639572 4833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:04.639828 4833 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:04.640152 4833 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:04.640189 4833 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:04.640220 4833 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:04.640239 4833 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:04.646250 4833 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.131:45515
I20260811 05:13:04.646306 5070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.131:45515 every 8 connection(s)
I20260811 05:13:04.646601 4833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
I20260811 05:13:04.647620 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 4833
I20260811 05:13:04.647740 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance
I20260811 05:13:04.647995 29246 external_mini_cluster.cc:1428] Setting key 58c5abb434dc2e0cb650dc199f8bd97e
Aug 11 05:13:04 dist-test-slave-f7th krb5kdc[4133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425184, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:13:04.656862 4827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.647384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:52757 (local address 127.28.143.190:32805)
0811 05:13:04.647479 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:04.647482 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:04.647736 (+ 254us) server_negotiation.cc:407] Connection header received
0811 05:13:04.648358 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:04.648360 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:04.648372 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:04.648426 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:04.649581 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.650095 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.651312 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.651539 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.651600 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:04.654176 (+ 2576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:04.654177 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:04.654178 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:04.654186 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:04.655298 (+ 1112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:04.655844 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:04.655846 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:04.655846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:04.655908 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:04.656158 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:04.656159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:04.656159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:04.656287 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:04.656350 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:04.656652 (+ 302us) server_negotiation.cc:299] Negotiation successful
0811 05:13:04.656763 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 05:13:04.657056 5077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.647344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:32805 (local address 127.28.143.131:52757)
0811 05:13:04.647574 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:04.647632 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:04.648263 (+ 631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:04.648515 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:04.648518 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:04.648926 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:04.649479 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:04.649485 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.650194 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.650195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:04.651196 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:04.651200 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.651622 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.651624 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:04.651739 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:04.652490 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:04.652500 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:04.654069 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:04.655385 (+ 1316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:04.655394 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:04.655397 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:04.655747 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:04.655971 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:04.655971 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:04.655972 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:04.656098 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:04.656348 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:04.656356 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:04.656506 (+ 150us) client_negotiation.cc:769] Sending connection context
0811 05:13:04.656602 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 05:13:04.656808 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":133,"threads_started":1}
I20260811 05:13:04.657615 5075 heartbeater.cc:347] Connected to a master server at 127.28.143.190:32805
I20260811 05:13:04.657724 5075 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:04.657915 5075 heartbeater.cc:511] Master 127.28.143.190:32805 requested a full tablet report, sending...
I20260811 05:13:04.658329 4207 ts_manager.cc:194] Registered new tserver with Master: 0bbaa1b956294d61a8d03a64257d5506 (127.28.143.131:45515)
I20260811 05:13:04.658726 4207 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.131@KRBTEST.COM'} at 127.28.143.131:52757
I20260811 05:13:04.659947 29246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:13:04.667709 4827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.661271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38032 (local address 127.28.143.190:32805)
0811 05:13:04.661339 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:04.661341 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:04.661434 (+ 93us) server_negotiation.cc:407] Connection header received
0811 05:13:04.661477 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:04.661477 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:04.661487 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:04.661540 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:04.662105 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.662613 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.663437 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.663634 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.663691 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:04.665009 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:04.665011 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:04.665011 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:04.665019 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:04.666263 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:04.666670 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:04.666672 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:04.666672 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:04.666737 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:04.666969 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:04.666970 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:04.666970 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:04.667141 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:04.667225 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:04.667515 (+ 290us) server_negotiation.cc:299] Negotiation successful
0811 05:13:04.667605 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 05:13:04.669454 4207 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38032:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:13:04.671057 4207 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:13:04.678104 5091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.673777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:35831 (local address 127.0.0.1:56796)
0811 05:13:04.674502 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:04.674511 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:04.674564 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:04.674807 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:04.674808 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:04.674810 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:04.674848 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:04.674850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.676058 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.676059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:04.677080 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:04.677082 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.677850 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.677851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:04.677922 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:04.677927 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 05:13:04.677980 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 05:13:04.678024 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":708,"thread_start_us":112,"threads_started":1}
I20260811 05:13:04.678130 5099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.674101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56796 (local address 127.28.143.129:35831)
0811 05:13:04.674237 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:04.674241 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:04.674618 (+ 377us) server_negotiation.cc:407] Connection header received
0811 05:13:04.674656 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:04.674658 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:04.674674 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:04.674741 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:04.674931 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.675967 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.677182 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.677760 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.677837 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:04.677894 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:04.678002 (+ 108us) server_negotiation.cc:299] Negotiation successful
0811 05:13:04.678052 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":73,"threads_started":1}
I20260811 05:13:04.678148 5104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.674094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36250 (local address 127.28.143.131:45515)
0811 05:13:04.674284 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:04.674289 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:04.674301 (+ 12us) server_negotiation.cc:407] Connection header received
0811 05:13:04.674343 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:04.674345 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:04.674361 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:04.674443 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:04.674731 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.675480 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.676785 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.677584 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.677657 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:04.677719 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:04.678027 (+ 308us) server_negotiation.cc:299] Negotiation successful
0811 05:13:04.678071 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":87,"threads_started":1}
I20260811 05:13:04.678218 5098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.673961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58452 (local address 127.28.143.130:35843)
0811 05:13:04.674151 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:04.674156 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:04.674342 (+ 186us) server_negotiation.cc:407] Connection header received
0811 05:13:04.674425 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:04.674427 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:04.674443 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:04.674520 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:04.674731 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.675827 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.677086 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.677781 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.677824 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:04.677870 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:04.678102 (+ 232us) server_negotiation.cc:299] Negotiation successful
0811 05:13:04.678143 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":87,"threads_started":1}
I20260811 05:13:04.678315 5093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.673769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.131:45515 (local address 127.0.0.1:36250)
0811 05:13:04.674018 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:04.674063 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:04.674135 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:04.674460 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:04.674463 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:04.674466 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:04.674549 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:04.674551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.675582 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.675583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:04.676702 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:04.676705 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.677683 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.677684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:04.677771 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:04.677788 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 05:13:04.677855 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 05:13:04.677919 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":120,"threads_started":1}
I20260811 05:13:04.678747 5092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.673771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:35843 (local address 127.0.0.1:58452)
0811 05:13:04.674275 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:04.674286 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:04.674348 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:04.674609 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:04.674610 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:04.674612 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:04.674649 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:04.674650 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.675943 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.675943 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:04.676970 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:04.676973 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.677962 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.677963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:04.678017 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:04.678020 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 05:13:04.678621 (+ 601us) client_negotiation.cc:240] Negotiation successful
0811 05:13:04.678671 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":118,"threads_started":1}
I20260811 05:13:04.679236 4942 tablet_service.cc:1467] Processing CreateTablet for tablet 5970cbc3e2ae43289a6237ed3d30a1f1 (DEFAULT_TABLE table=test-table [id=6c8320f44532486aabf506d8bd3453c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:04.679234 4438 tablet_service.cc:1467] Processing CreateTablet for tablet 5970cbc3e2ae43289a6237ed3d30a1f1 (DEFAULT_TABLE table=test-table [id=6c8320f44532486aabf506d8bd3453c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:04.679648 4942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5970cbc3e2ae43289a6237ed3d30a1f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:04.679648 4438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5970cbc3e2ae43289a6237ed3d30a1f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:04.683133 5106 tablet_bootstrap.cc:492] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 89a0a1577ee94a168f79c3edcebe0f08: Bootstrap starting.
I20260811 05:13:04.683976 5106 tablet_bootstrap.cc:654] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 89a0a1577ee94a168f79c3edcebe0f08: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:04.684000 4696 tablet_service.cc:1467] Processing CreateTablet for tablet 5970cbc3e2ae43289a6237ed3d30a1f1 (DEFAULT_TABLE table=test-table [id=6c8320f44532486aabf506d8bd3453c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:04.684345 5106 log.cc:824] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 89a0a1577ee94a168f79c3edcebe0f08: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:04.684379 4696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5970cbc3e2ae43289a6237ed3d30a1f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:04.684767 5107 tablet_bootstrap.cc:492] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 0bbaa1b956294d61a8d03a64257d5506: Bootstrap starting.
I20260811 05:13:04.685166 5106 tablet_bootstrap.cc:492] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 89a0a1577ee94a168f79c3edcebe0f08: No bootstrap required, opened a new log
I20260811 05:13:04.685236 5106 ts_tablet_manager.cc:1397] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 89a0a1577ee94a168f79c3edcebe0f08: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 05:13:04.685631 5107 tablet_bootstrap.cc:654] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 0bbaa1b956294d61a8d03a64257d5506: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:04.686290 5107 log.cc:824] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 0bbaa1b956294d61a8d03a64257d5506: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:04.687196 5107 tablet_bootstrap.cc:492] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 0bbaa1b956294d61a8d03a64257d5506: No bootstrap required, opened a new log
I20260811 05:13:04.687274 5107 ts_tablet_manager.cc:1397] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 0bbaa1b956294d61a8d03a64257d5506: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 05:13:04.687475 5106 raft_consensus.cc:348] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 89a0a1577ee94a168f79c3edcebe0f08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89a0a1577ee94a168f79c3edcebe0f08" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 35831 } } peers { permanent_uuid: "39fd45c64c71497b8226e52caaf3dbde" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 35843 } } peers { permanent_uuid: "0bbaa1b956294d61a8d03a64257d5506" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 45515 } }
I20260811 05:13:04.687618 5106 raft_consensus.cc:374] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 89a0a1577ee94a168f79c3edcebe0f08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:04.687652 5106 raft_consensus.cc:729] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 89a0a1577ee94a168f79c3edcebe0f08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89a0a1577ee94a168f79c3edcebe0f08, State: Initialized, Role: FOLLOWER
I20260811 05:13:04.687819 5106 consensus_queue.cc:260] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 89a0a1577ee94a168f79c3edcebe0f08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89a0a1577ee94a168f79c3edcebe0f08" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 35831 } } peers { permanent_uuid: "39fd45c64c71497b8226e52caaf3dbde" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 35843 } } peers { permanent_uuid: "0bbaa1b956294d61a8d03a64257d5506" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 45515 } }
I20260811 05:13:04.688113 5106 ts_tablet_manager.cc:1428] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 89a0a1577ee94a168f79c3edcebe0f08: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 05:13:04.688817 5113 tablet_bootstrap.cc:492] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 39fd45c64c71497b8226e52caaf3dbde: Bootstrap starting.
I20260811 05:13:04.689256 5107 raft_consensus.cc:348] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 0bbaa1b956294d61a8d03a64257d5506 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89a0a1577ee94a168f79c3edcebe0f08" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 35831 } } peers { permanent_uuid: "39fd45c64c71497b8226e52caaf3dbde" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 35843 } } peers { permanent_uuid: "0bbaa1b956294d61a8d03a64257d5506" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 45515 } }
I20260811 05:13:04.689426 5107 raft_consensus.cc:374] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 0bbaa1b956294d61a8d03a64257d5506 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:04.689504 5113 tablet_bootstrap.cc:654] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 39fd45c64c71497b8226e52caaf3dbde: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:04.689512 5107 raft_consensus.cc:729] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 0bbaa1b956294d61a8d03a64257d5506 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bbaa1b956294d61a8d03a64257d5506, State: Initialized, Role: FOLLOWER
I20260811 05:13:04.689642 5107 consensus_queue.cc:260] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 0bbaa1b956294d61a8d03a64257d5506 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89a0a1577ee94a168f79c3edcebe0f08" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 35831 } } peers { permanent_uuid: "39fd45c64c71497b8226e52caaf3dbde" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 35843 } } peers { permanent_uuid: "0bbaa1b956294d61a8d03a64257d5506" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 45515 } }
I20260811 05:13:04.690002 5075 heartbeater.cc:503] Master 127.28.143.190:32805 was elected leader, sending a full tablet report...
I20260811 05:13:04.689985 5107 ts_tablet_manager.cc:1428] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 0bbaa1b956294d61a8d03a64257d5506: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 05:13:04.690142 5113 log.cc:824] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 39fd45c64c71497b8226e52caaf3dbde: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:04.690953 5113 tablet_bootstrap.cc:492] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 39fd45c64c71497b8226e52caaf3dbde: No bootstrap required, opened a new log
I20260811 05:13:04.691042 5113 ts_tablet_manager.cc:1397] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 39fd45c64c71497b8226e52caaf3dbde: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 05:13:04.692463 5113 raft_consensus.cc:348] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 39fd45c64c71497b8226e52caaf3dbde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89a0a1577ee94a168f79c3edcebe0f08" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 35831 } } peers { permanent_uuid: "39fd45c64c71497b8226e52caaf3dbde" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 35843 } } peers { permanent_uuid: "0bbaa1b956294d61a8d03a64257d5506" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 45515 } }
I20260811 05:13:04.692606 5113 raft_consensus.cc:374] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 39fd45c64c71497b8226e52caaf3dbde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:04.692679 5113 raft_consensus.cc:729] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 39fd45c64c71497b8226e52caaf3dbde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39fd45c64c71497b8226e52caaf3dbde, State: Initialized, Role: FOLLOWER
I20260811 05:13:04.692800 5113 consensus_queue.cc:260] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 39fd45c64c71497b8226e52caaf3dbde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89a0a1577ee94a168f79c3edcebe0f08" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 35831 } } peers { permanent_uuid: "39fd45c64c71497b8226e52caaf3dbde" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 35843 } } peers { permanent_uuid: "0bbaa1b956294d61a8d03a64257d5506" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 45515 } }
I20260811 05:13:04.693102 5113 ts_tablet_manager.cc:1428] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 39fd45c64c71497b8226e52caaf3dbde: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 05:13:04.693141 4825 heartbeater.cc:503] Master 127.28.143.190:32805 was elected leader, sending a full tablet report...
W20260811 05:13:04.730881 4826 tablet.cc:2369] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 39fd45c64c71497b8226e52caaf3dbde: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 05:13:04.753393 4572 tablet.cc:2369] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 89a0a1577ee94a168f79c3edcebe0f08: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:13:04.765359 5114 raft_consensus.cc:482] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 89a0a1577ee94a168f79c3edcebe0f08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:13:04.765513 5114 raft_consensus.cc:504] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 89a0a1577ee94a168f79c3edcebe0f08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89a0a1577ee94a168f79c3edcebe0f08" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 35831 } } peers { permanent_uuid: "39fd45c64c71497b8226e52caaf3dbde" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 35843 } } peers { permanent_uuid: "0bbaa1b956294d61a8d03a64257d5506" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 45515 } }
I20260811 05:13:04.765906 5114 leader_election.cc:288] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 89a0a1577ee94a168f79c3edcebe0f08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39fd45c64c71497b8226e52caaf3dbde (127.28.143.130:35843), 0bbaa1b956294d61a8d03a64257d5506 (127.28.143.131:45515)
I20260811 05:13:04.769132 5124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.766155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.131:45515 (local address 127.28.143.129:34631)
0811 05:13:04.766374 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:04.766387 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:04.766455 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:04.766593 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:04.766594 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:04.766596 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:04.766672 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:04.766674 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.767307 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.767308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:04.768208 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:04.768211 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.768865 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.768866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:04.768938 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:04.768942 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:13:04.768989 (+ 47us) client_negotiation.cc:240] Negotiation successful
0811 05:13:04.769035 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":127,"threads_started":1}
I20260811 05:13:04.769120 5104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.766251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:34631 (local address 127.28.143.131:45515)
0811 05:13:04.766368 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:04.766370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:04.766451 (+ 81us) server_negotiation.cc:407] Connection header received
0811 05:13:04.766518 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:04.766519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:04.766530 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:04.766579 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:04.766731 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.767216 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.768318 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.768775 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.768847 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:04.768858 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:04.769007 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 05:13:04.769049 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 05:13:04.769232 5098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.766327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:33455 (local address 127.28.143.130:35843)
0811 05:13:04.766372 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:04.766374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:04.766500 (+ 126us) server_negotiation.cc:407] Connection header received
0811 05:13:04.766589 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:04.766590 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:04.766600 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:04.766647 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:04.766804 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.767284 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.768290 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.768872 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.768960 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:04.768970 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:04.769129 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 05:13:04.769167 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 05:13:04.769323 5122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.766157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:35843 (local address 127.28.143.129:33455)
0811 05:13:04.766437 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:04.766448 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:04.766527 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:04.766665 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:04.766666 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:04.766668 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:04.766738 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:04.766739 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.767375 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.767376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:04.768180 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:04.768183 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.768990 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.768991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:04.769062 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:04.769066 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:13:04.769116 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 05:13:04.769159 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":2,"thread_start_us":145,"threads_started":1}
I20260811 05:13:04.769691 4986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5970cbc3e2ae43289a6237ed3d30a1f1" candidate_uuid: "89a0a1577ee94a168f79c3edcebe0f08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bbaa1b956294d61a8d03a64257d5506" is_pre_election: true
I20260811 05:13:04.769737 4740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5970cbc3e2ae43289a6237ed3d30a1f1" candidate_uuid: "89a0a1577ee94a168f79c3edcebe0f08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39fd45c64c71497b8226e52caaf3dbde" is_pre_election: true
I20260811 05:13:04.769845 4986 raft_consensus.cc:2454] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 0bbaa1b956294d61a8d03a64257d5506 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89a0a1577ee94a168f79c3edcebe0f08 in term 0.
I20260811 05:13:04.769881 4740 raft_consensus.cc:2454] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 39fd45c64c71497b8226e52caaf3dbde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89a0a1577ee94a168f79c3edcebe0f08 in term 0.
I20260811 05:13:04.770148 4355 leader_election.cc:302] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 89a0a1577ee94a168f79c3edcebe0f08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bbaa1b956294d61a8d03a64257d5506, 89a0a1577ee94a168f79c3edcebe0f08; no voters:
I20260811 05:13:04.770346 5114 raft_consensus.cc:2781] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 89a0a1577ee94a168f79c3edcebe0f08 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:13:04.770422 5114 raft_consensus.cc:482] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 89a0a1577ee94a168f79c3edcebe0f08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:13:04.770449 5114 raft_consensus.cc:3037] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 89a0a1577ee94a168f79c3edcebe0f08 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:04.771176 5114 raft_consensus.cc:504] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 89a0a1577ee94a168f79c3edcebe0f08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89a0a1577ee94a168f79c3edcebe0f08" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 35831 } } peers { permanent_uuid: "39fd45c64c71497b8226e52caaf3dbde" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 35843 } } peers { permanent_uuid: "0bbaa1b956294d61a8d03a64257d5506" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 45515 } }
I20260811 05:13:04.771332 5114 leader_election.cc:288] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 89a0a1577ee94a168f79c3edcebe0f08 [CANDIDATE]: Term 1 election: Requested vote from peers 39fd45c64c71497b8226e52caaf3dbde (127.28.143.130:35843), 0bbaa1b956294d61a8d03a64257d5506 (127.28.143.131:45515)
I20260811 05:13:04.771540 4986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5970cbc3e2ae43289a6237ed3d30a1f1" candidate_uuid: "89a0a1577ee94a168f79c3edcebe0f08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bbaa1b956294d61a8d03a64257d5506"
I20260811 05:13:04.771555 4740 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5970cbc3e2ae43289a6237ed3d30a1f1" candidate_uuid: "89a0a1577ee94a168f79c3edcebe0f08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39fd45c64c71497b8226e52caaf3dbde"
I20260811 05:13:04.771625 4986 raft_consensus.cc:3037] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 0bbaa1b956294d61a8d03a64257d5506 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:04.771633 4740 raft_consensus.cc:3037] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 39fd45c64c71497b8226e52caaf3dbde [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:04.772394 4986 raft_consensus.cc:2454] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 0bbaa1b956294d61a8d03a64257d5506 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89a0a1577ee94a168f79c3edcebe0f08 in term 1.
I20260811 05:13:04.772706 4355 leader_election.cc:302] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 89a0a1577ee94a168f79c3edcebe0f08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bbaa1b956294d61a8d03a64257d5506, 89a0a1577ee94a168f79c3edcebe0f08; no voters:
I20260811 05:13:04.772822 5114 raft_consensus.cc:2781] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 89a0a1577ee94a168f79c3edcebe0f08 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:04.772706 4740 raft_consensus.cc:2454] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 39fd45c64c71497b8226e52caaf3dbde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89a0a1577ee94a168f79c3edcebe0f08 in term 1.
I20260811 05:13:04.773005 5114 raft_consensus.cc:686] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 89a0a1577ee94a168f79c3edcebe0f08 [term 1 LEADER]: Becoming Leader. State: Replica: 89a0a1577ee94a168f79c3edcebe0f08, State: Running, Role: LEADER
I20260811 05:13:04.773130 5114 consensus_queue.cc:237] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 89a0a1577ee94a168f79c3edcebe0f08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89a0a1577ee94a168f79c3edcebe0f08" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 35831 } } peers { permanent_uuid: "39fd45c64c71497b8226e52caaf3dbde" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 35843 } } peers { permanent_uuid: "0bbaa1b956294d61a8d03a64257d5506" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 45515 } }
I20260811 05:13:04.773965 4207 catalog_manager.cc:5495] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 89a0a1577ee94a168f79c3edcebe0f08 reported cstate change: term changed from 0 to 1, leader changed from <none> to 89a0a1577ee94a168f79c3edcebe0f08 (127.28.143.129). New cstate: current_term: 1 leader_uuid: "89a0a1577ee94a168f79c3edcebe0f08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89a0a1577ee94a168f79c3edcebe0f08" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 35831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39fd45c64c71497b8226e52caaf3dbde" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 35843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bbaa1b956294d61a8d03a64257d5506" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 45515 } health_report { overall_health: UNKNOWN } } }
I20260811 05:13:04.795097 5099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.792260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56812 (local address 127.28.143.129:35831)
0811 05:13:04.792353 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:04.792355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:04.792365 (+ 10us) server_negotiation.cc:407] Connection header received
0811 05:13:04.792405 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:04.792406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:04.792436 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:04.792505 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:13:04.792709 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.793243 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.794116 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:04.794322 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:04.794392 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:04.794670 (+ 278us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:13:04.794795 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:13:04.794931 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:04.794977 (+ 46us) server_negotiation.cc:299] Negotiation successful
0811 05:13:04.794999 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 05:13:04.797330 4740 raft_consensus.cc:1261] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 39fd45c64c71497b8226e52caaf3dbde [term 1 FOLLOWER]: Refusing update from remote peer 89a0a1577ee94a168f79c3edcebe0f08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:04.797319 4986 raft_consensus.cc:1261] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 0bbaa1b956294d61a8d03a64257d5506 [term 1 FOLLOWER]: Refusing update from remote peer 89a0a1577ee94a168f79c3edcebe0f08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:04.797753 5125 consensus_queue.cc:1035] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 89a0a1577ee94a168f79c3edcebe0f08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39fd45c64c71497b8226e52caaf3dbde" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 35843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:04.797905 5114 consensus_queue.cc:1035] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 89a0a1577ee94a168f79c3edcebe0f08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bbaa1b956294d61a8d03a64257d5506" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 45515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:04.800951 5129 mvcc.cc:205] Tried to move back new op lower bound from 7317197556926357504 to 7317197556831055872. Current Snapshot: MvccSnapshot[applied={T|T < 7317197556926357504}]
I20260811 05:13:04.801352 5131 mvcc.cc:205] Tried to move back new op lower bound from 7317197556926357504 to 7317197556831055872. Current Snapshot: MvccSnapshot[applied={T|T < 7317197556926357504}]
I20260811 05:13:04.801630 5138 mvcc.cc:205] Tried to move back new op lower bound from 7317197556926357504 to 7317197556831055872. Current Snapshot: MvccSnapshot[applied={T|T < 7317197556926357504}]
I20260811 05:13:04.803781 4207 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38032:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:04.803915 4207 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38032:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:04.804895 4207 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 218cebc95c51478fb9dc0ba1b1f40fde: Sending DeleteTablet for 3 replicas of tablet 5970cbc3e2ae43289a6237ed3d30a1f1
I20260811 05:13:04.805365 4696 tablet_service.cc:1514] Processing DeleteTablet for tablet 5970cbc3e2ae43289a6237ed3d30a1f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:04 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:58452
I20260811 05:13:04.805482 4438 tablet_service.cc:1514] Processing DeleteTablet for tablet 5970cbc3e2ae43289a6237ed3d30a1f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:04 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:56796
I20260811 05:13:04.805683 4942 tablet_service.cc:1514] Processing DeleteTablet for tablet 5970cbc3e2ae43289a6237ed3d30a1f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:04 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:36250
I20260811 05:13:04.806041 5151 tablet_replica.cc:330] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 0bbaa1b956294d61a8d03a64257d5506: stopping tablet replica
I20260811 05:13:04.806035 5148 tablet_replica.cc:330] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 39fd45c64c71497b8226e52caaf3dbde: stopping tablet replica
I20260811 05:13:04.806135 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 4320
I20260811 05:13:04.806176 5148 raft_consensus.cc:2229] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 39fd45c64c71497b8226e52caaf3dbde [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:04.806299 5148 raft_consensus.cc:2258] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 39fd45c64c71497b8226e52caaf3dbde [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:04.806224 5151 raft_consensus.cc:2229] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 0bbaa1b956294d61a8d03a64257d5506 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:04.813967 5151 raft_consensus.cc:2258] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 0bbaa1b956294d61a8d03a64257d5506 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:04.814815 5148 ts_tablet_manager.cc:1905] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 39fd45c64c71497b8226e52caaf3dbde: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:04.814815 5151 ts_tablet_manager.cc:1905] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 0bbaa1b956294d61a8d03a64257d5506: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:04.817597 5151 ts_tablet_manager.cc:1918] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 0bbaa1b956294d61a8d03a64257d5506: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:13:04.817597 5148 ts_tablet_manager.cc:1918] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 39fd45c64c71497b8226e52caaf3dbde: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:13:04.817719 5151 log.cc:1196] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 0bbaa1b956294d61a8d03a64257d5506: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-2/wal/wals/5970cbc3e2ae43289a6237ed3d30a1f1
I20260811 05:13:04.817719 5148 log.cc:1196] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 39fd45c64c71497b8226e52caaf3dbde: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425159060450-29246-0/minicluster-data/ts-1/wal/wals/5970cbc3e2ae43289a6237ed3d30a1f1
I20260811 05:13:04.818025 5148 ts_tablet_manager.cc:1939] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 39fd45c64c71497b8226e52caaf3dbde: Deleting consensus metadata
I20260811 05:13:04.818027 5151 ts_tablet_manager.cc:1939] T 5970cbc3e2ae43289a6237ed3d30a1f1 P 0bbaa1b956294d61a8d03a64257d5506: Deleting consensus metadata
I20260811 05:13:04.818606 4179 catalog_manager.cc:4841] TS 0bbaa1b956294d61a8d03a64257d5506 (127.28.143.131:45515): tablet 5970cbc3e2ae43289a6237ed3d30a1f1 (table test-table [id=6c8320f44532486aabf506d8bd3453c1]) successfully deleted
I20260811 05:13:04.818606 4182 catalog_manager.cc:4841] TS 39fd45c64c71497b8226e52caaf3dbde (127.28.143.130:35843): tablet 5970cbc3e2ae43289a6237ed3d30a1f1 (table test-table [id=6c8320f44532486aabf506d8bd3453c1]) successfully deleted
W20260811 05:13:04.818867 4180 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.143.129:35831 (error 108)
I20260811 05:13:04.819072 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 4582
I20260811 05:13:04.831749 5093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:04.831510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:35831 (local address 127.0.0.1:56816)
0811 05:13:04.831585 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:04.831649 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.129:35831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 05:13:04.831969 4180 catalog_manager.cc:4584] TS 89a0a1577ee94a168f79c3edcebe0f08 (127.28.143.129:35831): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5970cbc3e2ae43289a6237ed3d30a1f1: Network error: Client connection negotiation failed: client connection to 127.28.143.129:35831: connect: Connection refused (error 111)
I20260811 05:13:04.836187 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 4833
I20260811 05:13:04.847632 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 4151
2026-08-11T05:13:04Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (3964 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T05:13:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:13:04Z Disabled control of system clock
I20260811 05:13:04.881978 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:35193
--webserver_interface=127.28.143.190
--webserver_port=0
--builtin_ntp_servers=127.28.143.148:36243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:35193
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 05:13:04.974094 5162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:04.974332 5162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:04.974355 5162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:04.975636 5162 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:04.975685 5162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:04.975697 5162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:04.975709 5162 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:04.975720 5162 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:13:04.977260 5162 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:36243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:35193
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:35193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5162
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:04.977535 5162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:04.977859 5162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:04.983029 5173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:04.982997 5176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:04.983491 5171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:04.983847 5162 server_base.cc:1034] running on GCE node
I20260811 05:13:04.984187 5162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:04.984560 5162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:04.985769 5162 hybrid_clock.cc:648] HybridClock initialized: now 1786425184985738 us; error 49 us; skew 500 ppm
I20260811 05:13:04.987182 5162 webserver.cc:460] Webserver started at http://127.28.143.190:40613/ using document root <none> and password file <none>
I20260811 05:13:04.987447 5162 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:04.987501 5162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:04.987663 5162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:04.988819 5162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "bf84ef7f308045dd98d957f096bfc0cc"
format_stamp: "Formatted at 2026-08-11 05:13:04 on dist-test-slave-f7th"
server_key: "51283f7e17336a6406bc9d0b0604ee2f"
server_key_iv: "51b22ac4201cf540e2cadaf387c3a3b1"
server_key_version: "encryptionkey@0"
I20260811 05:13:04.989151 5162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "bf84ef7f308045dd98d957f096bfc0cc"
format_stamp: "Formatted at 2026-08-11 05:13:04 on dist-test-slave-f7th"
server_key: "51283f7e17336a6406bc9d0b0604ee2f"
server_key_iv: "51b22ac4201cf540e2cadaf387c3a3b1"
server_key_version: "encryptionkey@0"
I20260811 05:13:04.990437 5162 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 05:13:04.991251 5185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:04.991431 5162 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:04.991525 5162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "bf84ef7f308045dd98d957f096bfc0cc"
format_stamp: "Formatted at 2026-08-11 05:13:04 on dist-test-slave-f7th"
server_key: "51283f7e17336a6406bc9d0b0604ee2f"
server_key_iv: "51b22ac4201cf540e2cadaf387c3a3b1"
server_key_version: "encryptionkey@0"
I20260811 05:13:04.991613 5162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:04.995143 5162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:04.995440 5162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:04.995576 5162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:04.999758 5162 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:35193
I20260811 05:13:04.999846 5280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:35193 every 8 connection(s)
I20260811 05:13:05.000166 5162 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:13:05.000990 5281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:05.004292 5281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf84ef7f308045dd98d957f096bfc0cc: Bootstrap starting.
I20260811 05:13:05.005008 5281 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf84ef7f308045dd98d957f096bfc0cc: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:05.005277 5281 log.cc:824] T 00000000000000000000000000000000 P bf84ef7f308045dd98d957f096bfc0cc: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:05.006170 5281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf84ef7f308045dd98d957f096bfc0cc: No bootstrap required, opened a new log
I20260811 05:13:05.007961 5281 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf84ef7f308045dd98d957f096bfc0cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf84ef7f308045dd98d957f096bfc0cc" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 35193 } }
I20260811 05:13:05.008111 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 5162
I20260811 05:13:05.008131 5281 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf84ef7f308045dd98d957f096bfc0cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:05.008165 5281 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf84ef7f308045dd98d957f096bfc0cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf84ef7f308045dd98d957f096bfc0cc, State: Initialized, Role: FOLLOWER
I20260811 05:13:05.008185 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:13:05.008350 29246 external_mini_cluster.cc:1428] Setting key 7b0215543d19404e2c96b7212c2ec405
I20260811 05:13:05.008314 5281 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf84ef7f308045dd98d957f096bfc0cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf84ef7f308045dd98d957f096bfc0cc" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 35193 } }
I20260811 05:13:05.008399 5281 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf84ef7f308045dd98d957f096bfc0cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:13:05.008453 5281 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf84ef7f308045dd98d957f096bfc0cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:13:05.008507 5281 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf84ef7f308045dd98d957f096bfc0cc [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:05.009383 5281 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf84ef7f308045dd98d957f096bfc0cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf84ef7f308045dd98d957f096bfc0cc" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 35193 } }
I20260811 05:13:05.009510 5281 leader_election.cc:302] T 00000000000000000000000000000000 P bf84ef7f308045dd98d957f096bfc0cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf84ef7f308045dd98d957f096bfc0cc; no voters:
I20260811 05:13:05.009686 5281 leader_election.cc:288] T 00000000000000000000000000000000 P bf84ef7f308045dd98d957f096bfc0cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:13:05.009745 5288 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf84ef7f308045dd98d957f096bfc0cc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:05.009914 5288 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf84ef7f308045dd98d957f096bfc0cc [term 1 LEADER]: Becoming Leader. State: Replica: bf84ef7f308045dd98d957f096bfc0cc, State: Running, Role: LEADER
I20260811 05:13:05.010004 5281 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf84ef7f308045dd98d957f096bfc0cc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:13:05.010396 5288 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf84ef7f308045dd98d957f096bfc0cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf84ef7f308045dd98d957f096bfc0cc" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 35193 } }
I20260811 05:13:05.011055 5287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:05.008977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38292 (local address 127.28.143.190:35193)
0811 05:13:05.009242 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:05.009253 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:05.009294 (+ 41us) server_negotiation.cc:407] Connection header received
0811 05:13:05.009435 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:05.009444 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:05.009496 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:05.009660 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:05.010541 (+ 881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:05.010545 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:05.010546 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:13:05.010551 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:05.010601 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:13:05.010616 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:05.010685 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:05.010785 (+ 100us) server_negotiation.cc:299] Negotiation successful
0811 05:13:05.010853 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":108,"threads_started":1}
I20260811 05:13:05.011492 5289 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf84ef7f308045dd98d957f096bfc0cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf84ef7f308045dd98d957f096bfc0cc. Latest consensus state: current_term: 1 leader_uuid: "bf84ef7f308045dd98d957f096bfc0cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf84ef7f308045dd98d957f096bfc0cc" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 35193 } } }
I20260811 05:13:05.011816 5293 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf84ef7f308045dd98d957f096bfc0cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf84ef7f308045dd98d957f096bfc0cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf84ef7f308045dd98d957f096bfc0cc" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 35193 } } }
I20260811 05:13:05.012231 5293 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf84ef7f308045dd98d957f096bfc0cc [sys.catalog]: This master's current role is: LEADER
W20260811 05:13:05.012293 5322 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bf84ef7f308045dd98d957f096bfc0cc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:13:05.012348 5322 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:13:05.011920 5289 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf84ef7f308045dd98d957f096bfc0cc [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:05.012482 5327 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:13:05.012682 5327 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:13:05.014812 5327 catalog_manager.cc:1347] Generated new cluster ID: e3bd2857c18546aa89533e217884e20c
I20260811 05:13:05.014878 5327 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:13:05.023650 5327 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:13:05.023859 5327 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:13:05.030347 5327 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf84ef7f308045dd98d957f096bfc0cc: Generated new TSK 0
I20260811 05:13:05.030591 5327 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:13:05.039824 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.129:0
--local_ip_for_outbound_sockets=127.28.143.129
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:35193
--builtin_ntp_servers=127.28.143.148:36243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 05:13:05.133718 5333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:05.133970 5333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:05.134004 5333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:05.135300 5333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:05.135351 5333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:05.135389 5333 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.129
I20260811 05:13:05.136997 5333 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:36243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:35193
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5333
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.129
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:05.137274 5333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:05.137607 5333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:05.142943 5347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:05.142942 5343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:05.143091 5344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:05.143543 5333 server_base.cc:1034] running on GCE node
I20260811 05:13:05.143808 5333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:05.144094 5333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:05.145288 5333 hybrid_clock.cc:648] HybridClock initialized: now 1786425185145265 us; error 37 us; skew 500 ppm
I20260811 05:13:05.146718 5333 webserver.cc:460] Webserver started at http://127.28.143.129:37923/ using document root <none> and password file <none>
I20260811 05:13:05.147066 5333 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:05.147122 5333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:05.147265 5333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:05.148536 5333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-0/data/instance:
uuid: "513da2c177e348349c4801d8df3cbd7e"
format_stamp: "Formatted at 2026-08-11 05:13:05 on dist-test-slave-f7th"
server_key: "b1b4985d74267ce880d3e1dea93505e1"
server_key_iv: "e43efcfb91b4e31493344d5433c83de8"
server_key_version: "encryptionkey@0"
I20260811 05:13:05.148962 5333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance:
uuid: "513da2c177e348349c4801d8df3cbd7e"
format_stamp: "Formatted at 2026-08-11 05:13:05 on dist-test-slave-f7th"
server_key: "b1b4985d74267ce880d3e1dea93505e1"
server_key_iv: "e43efcfb91b4e31493344d5433c83de8"
server_key_version: "encryptionkey@0"
I20260811 05:13:05.150280 5333 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:05.151031 5361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:05.151175 5333 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 05:13:05.151257 5333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-0/wal
uuid: "513da2c177e348349c4801d8df3cbd7e"
format_stamp: "Formatted at 2026-08-11 05:13:05 on dist-test-slave-f7th"
server_key: "b1b4985d74267ce880d3e1dea93505e1"
server_key_iv: "e43efcfb91b4e31493344d5433c83de8"
server_key_version: "encryptionkey@0"
I20260811 05:13:05.151329 5333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:05.154989 5333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:05.155294 5333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:05.155422 5333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:05.155634 5333 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:05.155959 5333 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:05.155992 5333 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:05.156023 5333 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:05.156046 5333 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:05.162595 5333 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.129:37163
I20260811 05:13:05.162807 5560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.129:37163 every 8 connection(s)
I20260811 05:13:05.163051 5333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
I20260811 05:13:05.166407 5287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:05.163836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:50413 (local address 127.28.143.190:35193)
0811 05:13:05.163893 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:05.163895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:05.164290 (+ 395us) server_negotiation.cc:407] Connection header received
0811 05:13:05.164781 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:05.164783 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:05.164792 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:05.164848 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:05.166103 (+ 1255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:05.166104 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:05.166105 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:13:05.166109 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:05.166128 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:13:05.166132 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:05.166192 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:05.166313 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 05:13:05.166336 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 05:13:05.166615 5564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:05.163820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:35193 (local address 127.28.143.129:50413)
0811 05:13:05.164087 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:05.164127 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:05.164671 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:05.164907 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:05.164911 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:05.164920 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:05.165917 (+ 997us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:13:05.165917 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:05.165950 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:13:05.165980 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:05.166224 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:05.166232 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:05.166235 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 05:13:05.166320 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 05:13:05.166386 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":91,"threads_started":1}
I20260811 05:13:05.167217 5562 heartbeater.cc:347] Connected to a master server at 127.28.143.190:35193
I20260811 05:13:05.167335 5562 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:05.167372 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 5333
I20260811 05:13:05.167470 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance
I20260811 05:13:05.167523 5562 heartbeater.cc:511] Master 127.28.143.190:35193 requested a full tablet report, sending...
I20260811 05:13:05.167673 29246 external_mini_cluster.cc:1428] Setting key 9b9eb2775e0c56c2aaf9cbf4831f2fcb
I20260811 05:13:05.168118 5216 ts_manager.cc:194] Registered new tserver with Master: 513da2c177e348349c4801d8df3cbd7e (127.28.143.129:37163)
I20260811 05:13:05.168736 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.130:0
--local_ip_for_outbound_sockets=127.28.143.130
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:35193
--builtin_ntp_servers=127.28.143.148:36243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 05:13:05.273876 5566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:05.274194 5566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:05.274231 5566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:05.276299 5566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:05.276401 5566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:05.276453 5566 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.130
I20260811 05:13:05.278954 5566 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:36243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:35193
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.130
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:05.279295 5566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:05.279716 5566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:05.285135 5578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:05.285128 5575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:05.285221 5566 server_base.cc:1034] running on GCE node
W20260811 05:13:05.285130 5580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:05.285528 5566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:05.285755 5566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:05.286912 5566 hybrid_clock.cc:648] HybridClock initialized: now 1786425185286874 us; error 49 us; skew 500 ppm
I20260811 05:13:05.288199 5566 webserver.cc:460] Webserver started at http://127.28.143.130:45589/ using document root <none> and password file <none>
I20260811 05:13:05.288473 5566 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:05.288522 5566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:05.288662 5566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:05.289754 5566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-1/data/instance:
uuid: "ede3a6bd28b641ed99d02e817d287d20"
format_stamp: "Formatted at 2026-08-11 05:13:05 on dist-test-slave-f7th"
server_key: "09d772c2e149826b87fdb520ebb634bb"
server_key_iv: "3178ac04cd9a4c205756a542ae45f613"
server_key_version: "encryptionkey@0"
I20260811 05:13:05.290114 5566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance:
uuid: "ede3a6bd28b641ed99d02e817d287d20"
format_stamp: "Formatted at 2026-08-11 05:13:05 on dist-test-slave-f7th"
server_key: "09d772c2e149826b87fdb520ebb634bb"
server_key_iv: "3178ac04cd9a4c205756a542ae45f613"
server_key_version: "encryptionkey@0"
I20260811 05:13:05.291482 5566 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 05:13:05.292281 5587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:05.292475 5566 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 05:13:05.292552 5566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-1/wal
uuid: "ede3a6bd28b641ed99d02e817d287d20"
format_stamp: "Formatted at 2026-08-11 05:13:05 on dist-test-slave-f7th"
server_key: "09d772c2e149826b87fdb520ebb634bb"
server_key_iv: "3178ac04cd9a4c205756a542ae45f613"
server_key_version: "encryptionkey@0"
I20260811 05:13:05.292649 5566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:05.296083 5566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:05.296365 5566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:05.296500 5566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:05.296726 5566 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:05.297032 5566 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:05.297065 5566 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:05.297093 5566 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:05.297118 5566 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:05.302560 5566 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.130:34619
I20260811 05:13:05.302639 5794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.130:34619 every 8 connection(s)
I20260811 05:13:05.302922 5566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
I20260811 05:13:05.305962 5287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:05.303644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:55193 (local address 127.28.143.190:35193)
0811 05:13:05.303714 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:05.303717 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:05.303929 (+ 212us) server_negotiation.cc:407] Connection header received
0811 05:13:05.304356 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:05.304357 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:05.304363 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:05.304413 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:05.305618 (+ 1205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:05.305619 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:05.305620 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:13:05.305624 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:05.305642 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:13:05.305646 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:05.305706 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:05.305889 (+ 183us) server_negotiation.cc:299] Negotiation successful
0811 05:13:05.305905 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 05:13:05.306147 5801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:05.303607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:35193 (local address 127.28.143.130:55193)
0811 05:13:05.303801 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:05.303834 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:05.304277 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:05.304476 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:05.304479 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:05.304487 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:05.305435 (+ 948us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:13:05.305436 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:05.305466 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:13:05.305496 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:05.305741 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:05.305748 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:05.305755 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 05:13:05.305855 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 05:13:05.305944 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":94,"threads_started":1}
I20260811 05:13:05.306664 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 5566
I20260811 05:13:05.306656 5795 heartbeater.cc:347] Connected to a master server at 127.28.143.190:35193
I20260811 05:13:05.306744 5795 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:05.306761 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance
I20260811 05:13:05.306933 5795 heartbeater.cc:511] Master 127.28.143.190:35193 requested a full tablet report, sending...
I20260811 05:13:05.306953 29246 external_mini_cluster.cc:1428] Setting key 23fd58e8cb63a841add79f0ac19c1e91
I20260811 05:13:05.307327 5214 ts_manager.cc:194] Registered new tserver with Master: ede3a6bd28b641ed99d02e817d287d20 (127.28.143.130:34619)
I20260811 05:13:05.307884 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.131:0
--local_ip_for_outbound_sockets=127.28.143.131
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:35193
--builtin_ntp_servers=127.28.143.148:36243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 05:13:05.394124 5803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:05.394354 5803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:05.394383 5803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:05.395679 5803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:05.395731 5803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:05.395767 5803 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.131
I20260811 05:13:05.397289 5803 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:36243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:35193
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.131
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:05.397578 5803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:05.397886 5803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:05.403439 5810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:05.403441 5803 server_base.cc:1034] running on GCE node
W20260811 05:13:05.403609 5812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:05.403462 5809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:05.403977 5803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:05.404263 5803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:05.405433 5803 hybrid_clock.cc:648] HybridClock initialized: now 1786425185405421 us; error 30 us; skew 500 ppm
I20260811 05:13:05.406919 5803 webserver.cc:460] Webserver started at http://127.28.143.131:46467/ using document root <none> and password file <none>
I20260811 05:13:05.407136 5803 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:05.407179 5803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:05.407274 5803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:05.408378 5803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-2/data/instance:
uuid: "cc0681329f1040af95adffe6df9b4b75"
format_stamp: "Formatted at 2026-08-11 05:13:05 on dist-test-slave-f7th"
server_key: "360f915941d97032c6b4db5c1d6f8e78"
server_key_iv: "8e8a0adf241e50e0d744e356c5c4c54b"
server_key_version: "encryptionkey@0"
I20260811 05:13:05.408733 5803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance:
uuid: "cc0681329f1040af95adffe6df9b4b75"
format_stamp: "Formatted at 2026-08-11 05:13:05 on dist-test-slave-f7th"
server_key: "360f915941d97032c6b4db5c1d6f8e78"
server_key_iv: "8e8a0adf241e50e0d744e356c5c4c54b"
server_key_version: "encryptionkey@0"
I20260811 05:13:05.410094 5803 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 05:13:05.410933 5828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:05.411194 5803 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 05:13:05.411288 5803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-2/wal
uuid: "cc0681329f1040af95adffe6df9b4b75"
format_stamp: "Formatted at 2026-08-11 05:13:05 on dist-test-slave-f7th"
server_key: "360f915941d97032c6b4db5c1d6f8e78"
server_key_iv: "8e8a0adf241e50e0d744e356c5c4c54b"
server_key_version: "encryptionkey@0"
I20260811 05:13:05.411444 5803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:05.415239 5803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:05.415505 5803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:05.415627 5803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:05.415825 5803 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:05.416152 5803 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:05.416186 5803 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:05.416217 5803 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:05.416240 5803 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:05.422047 5803 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.131:46537
I20260811 05:13:05.422102 6047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.131:46537 every 8 connection(s)
I20260811 05:13:05.422438 5803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
I20260811 05:13:05.423801 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 5803
I20260811 05:13:05.423884 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance
I20260811 05:13:05.424082 29246 external_mini_cluster.cc:1428] Setting key 1c25bb736bf35a18ec9ef1763745a452
I20260811 05:13:05.425760 5287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:05.423244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:42085 (local address 127.28.143.190:35193)
0811 05:13:05.423312 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:05.423314 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:05.423553 (+ 239us) server_negotiation.cc:407] Connection header received
0811 05:13:05.424090 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:05.424091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:05.424098 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:05.424167 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:05.425488 (+ 1321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:05.425489 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:05.425489 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:13:05.425494 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:05.425510 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:13:05.425514 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:05.425571 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:05.425692 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 05:13:05.425705 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 05:13:05.425963 6055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:05.423206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:35193 (local address 127.28.143.131:42085)
0811 05:13:05.423429 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:05.423473 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:05.423974 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:05.424257 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:05.424261 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:05.424269 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:05.425329 (+ 1060us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:13:05.425329 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:05.425363 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:13:05.425393 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:05.425609 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:05.425616 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:05.425618 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 05:13:05.425699 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 05:13:05.425753 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":84,"threads_started":1}
I20260811 05:13:05.426487 6052 heartbeater.cc:347] Connected to a master server at 127.28.143.190:35193
I20260811 05:13:05.426579 6052 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:05.426728 6052 heartbeater.cc:511] Master 127.28.143.190:35193 requested a full tablet report, sending...
I20260811 05:13:05.427155 5214 ts_manager.cc:194] Registered new tserver with Master: cc0681329f1040af95adffe6df9b4b75 (127.28.143.131:46537)
I20260811 05:13:05.436384 29246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:13:05.439702 5287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:05.438211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38296 (local address 127.28.143.190:35193)
0811 05:13:05.438299 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:05.438301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:05.438476 (+ 175us) server_negotiation.cc:407] Connection header received
0811 05:13:05.438511 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:05.438512 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:05.438519 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:05.438586 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:05.439393 (+ 807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:05.439395 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:05.439395 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:13:05.439399 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:05.439419 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:13:05.439423 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:05.439494 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:05.439596 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 05:13:05.439611 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 05:13:05.441813 5214 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38296:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:13:05.443552 5214 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:13:05.448683 6066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:05.446476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:37163 (local address 127.0.0.1:39994)
0811 05:13:05.446781 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:05.446790 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:05.446825 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:05.447156 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:05.447162 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:05.447169 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:05.448160 (+ 991us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:13:05.448160 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:05.448180 (+ 20us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:13:05.448205 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:05.448469 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:05.448477 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:05.448481 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:13:05.448553 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 05:13:05.448567 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":65,"threads_started":1}
I20260811 05:13:05.448832 6072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:05.446743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39994 (local address 127.28.143.129:37163)
0811 05:13:05.447031 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:05.447035 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:05.447043 (+ 8us) server_negotiation.cc:407] Connection header received
0811 05:13:05.447071 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:05.447073 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:05.447080 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:05.447140 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:05.448307 (+ 1167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:05.448320 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:05.448320 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:13:05.448325 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:05.448366 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:13:05.448400 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:05.448454 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:05.448690 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 05:13:05.448736 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":105,"threads_started":1}
I20260811 05:13:05.448873 6068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:05.446476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.131:46537 (local address 127.0.0.1:54790)
0811 05:13:05.446882 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:05.446890 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:05.446920 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:05.447226 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:05.447227 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:05.447231 (+ 4us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:05.448281 (+ 1050us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:13:05.448282 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:05.448292 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:13:05.448306 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:05.448725 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:05.448731 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:05.448732 (+ 1us) client_negotiation.cc:769] Sending connection context
0811 05:13:05.448796 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 05:13:05.448809 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"mutex_wait_us":2,"thread_start_us":115,"threads_started":1}
I20260811 05:13:05.449059 6074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:05.446743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54790 (local address 127.28.143.131:46537)
0811 05:13:05.446964 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:05.446969 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:05.446981 (+ 12us) server_negotiation.cc:407] Connection header received
0811 05:13:05.447009 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:05.447012 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:05.447066 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:05.447399 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:05.448530 (+ 1131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:05.448538 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:05.448538 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:13:05.448544 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:05.448606 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:13:05.448660 (+ 54us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:05.448719 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:05.448954 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 05:13:05.448972 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":107,"threads_started":1}
I20260811 05:13:05.449051 6070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:05.446473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:34619 (local address 127.0.0.1:54358)
0811 05:13:05.446676 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:05.446722 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:05.446782 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:05.447635 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:05.447636 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:05.447639 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:05.448419 (+ 780us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:13:05.448420 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:05.448427 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:13:05.448437 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:05.448952 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:05.448953 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:05.448954 (+ 1us) client_negotiation.cc:769] Sending connection context
0811 05:13:05.448979 (+ 25us) client_negotiation.cc:240] Negotiation successful
0811 05:13:05.448992 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":114,"threads_started":1}
I20260811 05:13:05.449542 6071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:05.446597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54358 (local address 127.28.143.130:34619)
0811 05:13:05.447502 (+ 905us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:05.447506 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:05.447516 (+ 10us) server_negotiation.cc:407] Connection header received
0811 05:13:05.447541 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:05.447543 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:05.447551 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:05.447623 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:05.448796 (+ 1173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:05.448807 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:05.448807 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:13:05.448813 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:05.448852 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:13:05.448881 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:05.449123 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:05.449436 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 05:13:05.449455 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":869,"thread_start_us":96,"threads_started":1}
I20260811 05:13:05.449887 5438 tablet_service.cc:1467] Processing CreateTablet for tablet e0a99b19956b4bb8b366c05032e3f0c9 (DEFAULT_TABLE table=test-table [id=f49591501e754583ae92b3a98235108c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:05.449895 5915 tablet_service.cc:1467] Processing CreateTablet for tablet e0a99b19956b4bb8b366c05032e3f0c9 (DEFAULT_TABLE table=test-table [id=f49591501e754583ae92b3a98235108c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:05.450311 5438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0a99b19956b4bb8b366c05032e3f0c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:05.450311 5915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0a99b19956b4bb8b366c05032e3f0c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:05.450260 5675 tablet_service.cc:1467] Processing CreateTablet for tablet e0a99b19956b4bb8b366c05032e3f0c9 (DEFAULT_TABLE table=test-table [id=f49591501e754583ae92b3a98235108c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:05.450587 5675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0a99b19956b4bb8b366c05032e3f0c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:05.453688 6077 tablet_bootstrap.cc:492] T e0a99b19956b4bb8b366c05032e3f0c9 P ede3a6bd28b641ed99d02e817d287d20: Bootstrap starting.
I20260811 05:13:05.453725 6079 tablet_bootstrap.cc:492] T e0a99b19956b4bb8b366c05032e3f0c9 P 513da2c177e348349c4801d8df3cbd7e: Bootstrap starting.
I20260811 05:13:05.454443 6079 tablet_bootstrap.cc:654] T e0a99b19956b4bb8b366c05032e3f0c9 P 513da2c177e348349c4801d8df3cbd7e: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:05.454443 6077 tablet_bootstrap.cc:654] T e0a99b19956b4bb8b366c05032e3f0c9 P ede3a6bd28b641ed99d02e817d287d20: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:05.454818 6079 log.cc:824] T e0a99b19956b4bb8b366c05032e3f0c9 P 513da2c177e348349c4801d8df3cbd7e: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:05.454820 6077 log.cc:824] T e0a99b19956b4bb8b366c05032e3f0c9 P ede3a6bd28b641ed99d02e817d287d20: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:05.455233 6080 tablet_bootstrap.cc:492] T e0a99b19956b4bb8b366c05032e3f0c9 P cc0681329f1040af95adffe6df9b4b75: Bootstrap starting.
I20260811 05:13:05.455709 6077 tablet_bootstrap.cc:492] T e0a99b19956b4bb8b366c05032e3f0c9 P ede3a6bd28b641ed99d02e817d287d20: No bootstrap required, opened a new log
I20260811 05:13:05.455709 6079 tablet_bootstrap.cc:492] T e0a99b19956b4bb8b366c05032e3f0c9 P 513da2c177e348349c4801d8df3cbd7e: No bootstrap required, opened a new log
I20260811 05:13:05.455790 6077 ts_tablet_manager.cc:1397] T e0a99b19956b4bb8b366c05032e3f0c9 P ede3a6bd28b641ed99d02e817d287d20: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 05:13:05.455790 6079 ts_tablet_manager.cc:1397] T e0a99b19956b4bb8b366c05032e3f0c9 P 513da2c177e348349c4801d8df3cbd7e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 05:13:05.455822 6080 tablet_bootstrap.cc:654] T e0a99b19956b4bb8b366c05032e3f0c9 P cc0681329f1040af95adffe6df9b4b75: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:05.456101 6080 log.cc:824] T e0a99b19956b4bb8b366c05032e3f0c9 P cc0681329f1040af95adffe6df9b4b75: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:05.456849 6080 tablet_bootstrap.cc:492] T e0a99b19956b4bb8b366c05032e3f0c9 P cc0681329f1040af95adffe6df9b4b75: No bootstrap required, opened a new log
I20260811 05:13:05.456914 6080 ts_tablet_manager.cc:1397] T e0a99b19956b4bb8b366c05032e3f0c9 P cc0681329f1040af95adffe6df9b4b75: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 05:13:05.458573 6077 raft_consensus.cc:348] T e0a99b19956b4bb8b366c05032e3f0c9 P ede3a6bd28b641ed99d02e817d287d20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513da2c177e348349c4801d8df3cbd7e" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37163 } } peers { permanent_uuid: "ede3a6bd28b641ed99d02e817d287d20" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34619 } } peers { permanent_uuid: "cc0681329f1040af95adffe6df9b4b75" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 46537 } }
I20260811 05:13:05.458781 6077 raft_consensus.cc:374] T e0a99b19956b4bb8b366c05032e3f0c9 P ede3a6bd28b641ed99d02e817d287d20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:05.458859 6077 raft_consensus.cc:729] T e0a99b19956b4bb8b366c05032e3f0c9 P ede3a6bd28b641ed99d02e817d287d20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ede3a6bd28b641ed99d02e817d287d20, State: Initialized, Role: FOLLOWER
I20260811 05:13:05.458845 6079 raft_consensus.cc:348] T e0a99b19956b4bb8b366c05032e3f0c9 P 513da2c177e348349c4801d8df3cbd7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513da2c177e348349c4801d8df3cbd7e" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37163 } } peers { permanent_uuid: "ede3a6bd28b641ed99d02e817d287d20" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34619 } } peers { permanent_uuid: "cc0681329f1040af95adffe6df9b4b75" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 46537 } }
I20260811 05:13:05.459029 6079 raft_consensus.cc:374] T e0a99b19956b4bb8b366c05032e3f0c9 P 513da2c177e348349c4801d8df3cbd7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:05.459102 6079 raft_consensus.cc:729] T e0a99b19956b4bb8b366c05032e3f0c9 P 513da2c177e348349c4801d8df3cbd7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 513da2c177e348349c4801d8df3cbd7e, State: Initialized, Role: FOLLOWER
I20260811 05:13:05.458987 6080 raft_consensus.cc:348] T e0a99b19956b4bb8b366c05032e3f0c9 P cc0681329f1040af95adffe6df9b4b75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513da2c177e348349c4801d8df3cbd7e" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37163 } } peers { permanent_uuid: "ede3a6bd28b641ed99d02e817d287d20" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34619 } } peers { permanent_uuid: "cc0681329f1040af95adffe6df9b4b75" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 46537 } }
I20260811 05:13:05.459077 6077 consensus_queue.cc:260] T e0a99b19956b4bb8b366c05032e3f0c9 P ede3a6bd28b641ed99d02e817d287d20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513da2c177e348349c4801d8df3cbd7e" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37163 } } peers { permanent_uuid: "ede3a6bd28b641ed99d02e817d287d20" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34619 } } peers { permanent_uuid: "cc0681329f1040af95adffe6df9b4b75" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 46537 } }
I20260811 05:13:05.459156 6080 raft_consensus.cc:374] T e0a99b19956b4bb8b366c05032e3f0c9 P cc0681329f1040af95adffe6df9b4b75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:05.459223 6080 raft_consensus.cc:729] T e0a99b19956b4bb8b366c05032e3f0c9 P cc0681329f1040af95adffe6df9b4b75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc0681329f1040af95adffe6df9b4b75, State: Initialized, Role: FOLLOWER
I20260811 05:13:05.459272 6079 consensus_queue.cc:260] T e0a99b19956b4bb8b366c05032e3f0c9 P 513da2c177e348349c4801d8df3cbd7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513da2c177e348349c4801d8df3cbd7e" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37163 } } peers { permanent_uuid: "ede3a6bd28b641ed99d02e817d287d20" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34619 } } peers { permanent_uuid: "cc0681329f1040af95adffe6df9b4b75" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 46537 } }
I20260811 05:13:05.459348 6080 consensus_queue.cc:260] T e0a99b19956b4bb8b366c05032e3f0c9 P cc0681329f1040af95adffe6df9b4b75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513da2c177e348349c4801d8df3cbd7e" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37163 } } peers { permanent_uuid: "ede3a6bd28b641ed99d02e817d287d20" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34619 } } peers { permanent_uuid: "cc0681329f1040af95adffe6df9b4b75" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 46537 } }
I20260811 05:13:05.459391 6077 ts_tablet_manager.cc:1428] T e0a99b19956b4bb8b366c05032e3f0c9 P ede3a6bd28b641ed99d02e817d287d20: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 05:13:05.459522 5795 heartbeater.cc:503] Master 127.28.143.190:35193 was elected leader, sending a full tablet report...
I20260811 05:13:05.459551 6079 ts_tablet_manager.cc:1428] T e0a99b19956b4bb8b366c05032e3f0c9 P 513da2c177e348349c4801d8df3cbd7e: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260811 05:13:05.459625 6080 ts_tablet_manager.cc:1428] T e0a99b19956b4bb8b366c05032e3f0c9 P cc0681329f1040af95adffe6df9b4b75: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 05:13:05.459676 5562 heartbeater.cc:503] Master 127.28.143.190:35193 was elected leader, sending a full tablet report...
I20260811 05:13:05.459794 6052 heartbeater.cc:503] Master 127.28.143.190:35193 was elected leader, sending a full tablet report...
I20260811 05:13:05.491709 6086 raft_consensus.cc:482] T e0a99b19956b4bb8b366c05032e3f0c9 P 513da2c177e348349c4801d8df3cbd7e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:13:05.491860 6086 raft_consensus.cc:504] T e0a99b19956b4bb8b366c05032e3f0c9 P 513da2c177e348349c4801d8df3cbd7e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513da2c177e348349c4801d8df3cbd7e" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37163 } } peers { permanent_uuid: "ede3a6bd28b641ed99d02e817d287d20" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34619 } } peers { permanent_uuid: "cc0681329f1040af95adffe6df9b4b75" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 46537 } }
I20260811 05:13:05.492300 6086 leader_election.cc:288] T e0a99b19956b4bb8b366c05032e3f0c9 P 513da2c177e348349c4801d8df3cbd7e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ede3a6bd28b641ed99d02e817d287d20 (127.28.143.130:34619), cc0681329f1040af95adffe6df9b4b75 (127.28.143.131:46537)
I20260811 05:13:05.494131 6074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:05.492777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:50423 (local address 127.28.143.131:46537)
0811 05:13:05.492832 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:05.492834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:05.492842 (+ 8us) server_negotiation.cc:407] Connection header received
0811 05:13:05.492909 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:05.492910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:05.492917 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:05.492976 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:05.493901 (+ 925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:05.493903 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:05.493903 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:13:05.493907 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:05.493925 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:13:05.493930 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:05.493975 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:05.494056 (+ 81us) server_negotiation.cc:299] Negotiation successful
0811 05:13:05.494069 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 05:13:05.494132 6071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:05.492730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:41193 (local address 127.28.143.130:34619)
0811 05:13:05.492775 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:05.492777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:05.492868 (+ 91us) server_negotiation.cc:407] Connection header received
0811 05:13:05.492934 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:05.492935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:05.492939 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:05.492993 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:05.493860 (+ 867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:05.493862 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:05.493862 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:13:05.493869 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:05.493890 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:13:05.493895 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:05.493951 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:05.494044 (+ 93us) server_negotiation.cc:299] Negotiation successful
0811 05:13:05.494062 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 05:13:05.494127 5564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:05.492560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.131:46537 (local address 127.28.143.129:50423)
0811 05:13:05.492669 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:05.492683 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:05.492745 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:05.493000 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:05.493001 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:05.493004 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:05.493778 (+ 774us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:13:05.493779 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:05.493787 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:13:05.493803 (+ 16us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:05.493988 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:05.493989 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:05.493990 (+ 1us) client_negotiation.cc:769] Sending connection context
0811 05:13:05.494036 (+ 46us) client_negotiation.cc:240] Negotiation successful
0811 05:13:05.494051 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 05:13:05.494369 6092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:05.492560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:34619 (local address 127.28.143.129:41193)
0811 05:13:05.492789 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:05.492799 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:05.492855 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:05.493000 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:05.493001 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:05.493004 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:05.493713 (+ 709us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:13:05.493713 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:05.493723 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:13:05.493740 (+ 17us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:05.493957 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:05.493958 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:05.493959 (+ 1us) client_negotiation.cc:769] Sending connection context
0811 05:13:05.494266 (+ 307us) client_negotiation.cc:240] Negotiation successful
0811 05:13:05.494283 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":19,"thread_start_us":145,"threads_started":1}
I20260811 05:13:05.494551 5962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0a99b19956b4bb8b366c05032e3f0c9" candidate_uuid: "513da2c177e348349c4801d8df3cbd7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc0681329f1040af95adffe6df9b4b75" is_pre_election: true
I20260811 05:13:05.494727 5962 raft_consensus.cc:2454] T e0a99b19956b4bb8b366c05032e3f0c9 P cc0681329f1040af95adffe6df9b4b75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 513da2c177e348349c4801d8df3cbd7e in term 0.
I20260811 05:13:05.494693 5711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0a99b19956b4bb8b366c05032e3f0c9" candidate_uuid: "513da2c177e348349c4801d8df3cbd7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ede3a6bd28b641ed99d02e817d287d20" is_pre_election: true
I20260811 05:13:05.494844 5711 raft_consensus.cc:2454] T e0a99b19956b4bb8b366c05032e3f0c9 P ede3a6bd28b641ed99d02e817d287d20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 513da2c177e348349c4801d8df3cbd7e in term 0.
I20260811 05:13:05.494987 5363 leader_election.cc:302] T e0a99b19956b4bb8b366c05032e3f0c9 P 513da2c177e348349c4801d8df3cbd7e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 513da2c177e348349c4801d8df3cbd7e, cc0681329f1040af95adffe6df9b4b75; no voters:
I20260811 05:13:05.495186 6086 raft_consensus.cc:2781] T e0a99b19956b4bb8b366c05032e3f0c9 P 513da2c177e348349c4801d8df3cbd7e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:13:05.495242 6086 raft_consensus.cc:482] T e0a99b19956b4bb8b366c05032e3f0c9 P 513da2c177e348349c4801d8df3cbd7e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:13:05.495265 6086 raft_consensus.cc:3037] T e0a99b19956b4bb8b366c05032e3f0c9 P 513da2c177e348349c4801d8df3cbd7e [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:05.496043 6086 raft_consensus.cc:504] T e0a99b19956b4bb8b366c05032e3f0c9 P 513da2c177e348349c4801d8df3cbd7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513da2c177e348349c4801d8df3cbd7e" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37163 } } peers { permanent_uuid: "ede3a6bd28b641ed99d02e817d287d20" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34619 } } peers { permanent_uuid: "cc0681329f1040af95adffe6df9b4b75" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 46537 } }
I20260811 05:13:05.496184 6086 leader_election.cc:288] T e0a99b19956b4bb8b366c05032e3f0c9 P 513da2c177e348349c4801d8df3cbd7e [CANDIDATE]: Term 1 election: Requested vote from peers ede3a6bd28b641ed99d02e817d287d20 (127.28.143.130:34619), cc0681329f1040af95adffe6df9b4b75 (127.28.143.131:46537)
I20260811 05:13:05.496367 5962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0a99b19956b4bb8b366c05032e3f0c9" candidate_uuid: "513da2c177e348349c4801d8df3cbd7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc0681329f1040af95adffe6df9b4b75"
I20260811 05:13:05.496381 5711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0a99b19956b4bb8b366c05032e3f0c9" candidate_uuid: "513da2c177e348349c4801d8df3cbd7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ede3a6bd28b641ed99d02e817d287d20"
I20260811 05:13:05.496450 5962 raft_consensus.cc:3037] T e0a99b19956b4bb8b366c05032e3f0c9 P cc0681329f1040af95adffe6df9b4b75 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:05.496469 5711 raft_consensus.cc:3037] T e0a99b19956b4bb8b366c05032e3f0c9 P ede3a6bd28b641ed99d02e817d287d20 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:05.497316 5962 raft_consensus.cc:2454] T e0a99b19956b4bb8b366c05032e3f0c9 P cc0681329f1040af95adffe6df9b4b75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 513da2c177e348349c4801d8df3cbd7e in term 1.
I20260811 05:13:05.497336 5711 raft_consensus.cc:2454] T e0a99b19956b4bb8b366c05032e3f0c9 P ede3a6bd28b641ed99d02e817d287d20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 513da2c177e348349c4801d8df3cbd7e in term 1.
I20260811 05:13:05.497504 5363 leader_election.cc:302] T e0a99b19956b4bb8b366c05032e3f0c9 P 513da2c177e348349c4801d8df3cbd7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 513da2c177e348349c4801d8df3cbd7e, cc0681329f1040af95adffe6df9b4b75; no voters:
I20260811 05:13:05.497645 6086 raft_consensus.cc:2781] T e0a99b19956b4bb8b366c05032e3f0c9 P 513da2c177e348349c4801d8df3cbd7e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:05.497818 6086 raft_consensus.cc:686] T e0a99b19956b4bb8b366c05032e3f0c9 P 513da2c177e348349c4801d8df3cbd7e [term 1 LEADER]: Becoming Leader. State: Replica: 513da2c177e348349c4801d8df3cbd7e, State: Running, Role: LEADER
I20260811 05:13:05.497905 6086 consensus_queue.cc:237] T e0a99b19956b4bb8b366c05032e3f0c9 P 513da2c177e348349c4801d8df3cbd7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513da2c177e348349c4801d8df3cbd7e" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37163 } } peers { permanent_uuid: "ede3a6bd28b641ed99d02e817d287d20" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34619 } } peers { permanent_uuid: "cc0681329f1040af95adffe6df9b4b75" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 46537 } }
I20260811 05:13:05.499006 5213 catalog_manager.cc:5495] T e0a99b19956b4bb8b366c05032e3f0c9 P 513da2c177e348349c4801d8df3cbd7e reported cstate change: term changed from 0 to 1, leader changed from <none> to 513da2c177e348349c4801d8df3cbd7e (127.28.143.129). New cstate: current_term: 1 leader_uuid: "513da2c177e348349c4801d8df3cbd7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513da2c177e348349c4801d8df3cbd7e" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ede3a6bd28b641ed99d02e817d287d20" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc0681329f1040af95adffe6df9b4b75" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 46537 } health_report { overall_health: UNKNOWN } } }
I20260811 05:13:05.509856 6072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:05.508780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40002 (local address 127.28.143.129:37163)
0811 05:13:05.508850 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:05.508852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:05.508924 (+ 72us) server_negotiation.cc:407] Connection header received
0811 05:13:05.508964 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:05.508965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:05.508982 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:05.509025 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:05.509590 (+ 565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:05.509591 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:05.509592 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:13:05.509597 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:05.509615 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:13:05.509619 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:05.509667 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:05.509724 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 05:13:05.509738 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 05:13:05.512034 5962 raft_consensus.cc:1261] T e0a99b19956b4bb8b366c05032e3f0c9 P cc0681329f1040af95adffe6df9b4b75 [term 1 FOLLOWER]: Refusing update from remote peer 513da2c177e348349c4801d8df3cbd7e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:05.512025 5711 raft_consensus.cc:1261] T e0a99b19956b4bb8b366c05032e3f0c9 P ede3a6bd28b641ed99d02e817d287d20 [term 1 FOLLOWER]: Refusing update from remote peer 513da2c177e348349c4801d8df3cbd7e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:05.512432 6093 consensus_queue.cc:1035] T e0a99b19956b4bb8b366c05032e3f0c9 P 513da2c177e348349c4801d8df3cbd7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc0681329f1040af95adffe6df9b4b75" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 46537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:05.512595 6086 consensus_queue.cc:1035] T e0a99b19956b4bb8b366c05032e3f0c9 P 513da2c177e348349c4801d8df3cbd7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ede3a6bd28b641ed99d02e817d287d20" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:05.515324 6100 mvcc.cc:205] Tried to move back new op lower bound from 7317197559853793280 to 7317197559800205312. Current Snapshot: MvccSnapshot[applied={T|T < 7317197559853793280}]
I20260811 05:13:05.515508 6102 mvcc.cc:205] Tried to move back new op lower bound from 7317197559853793280 to 7317197559800205312. Current Snapshot: MvccSnapshot[applied={T|T < 7317197559853793280}]
I20260811 05:13:05.517372 6104 mvcc.cc:205] Tried to move back new op lower bound from 7317197559853793280 to 7317197559800205312. Current Snapshot: MvccSnapshot[applied={T|T < 7317197559853793280 or (T in {7317197559853793280})}]
I20260811 05:13:05.518260 5214 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38296:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:05.518352 5214 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38296:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:05.519343 5214 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bf84ef7f308045dd98d957f096bfc0cc: Sending DeleteTablet for 3 replicas of tablet e0a99b19956b4bb8b366c05032e3f0c9
I20260811 05:13:05.519775 5675 tablet_service.cc:1514] Processing DeleteTablet for tablet e0a99b19956b4bb8b366c05032e3f0c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:05 UTC) from {username='slave'} at 127.0.0.1:54358
I20260811 05:13:05.520094 6116 tablet_replica.cc:330] T e0a99b19956b4bb8b366c05032e3f0c9 P ede3a6bd28b641ed99d02e817d287d20: stopping tablet replica
I20260811 05:13:05.520107 5915 tablet_service.cc:1514] Processing DeleteTablet for tablet e0a99b19956b4bb8b366c05032e3f0c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:05 UTC) from {username='slave'} at 127.0.0.1:54790
I20260811 05:13:05.520216 6116 raft_consensus.cc:2229] T e0a99b19956b4bb8b366c05032e3f0c9 P ede3a6bd28b641ed99d02e817d287d20 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:05.520335 6116 raft_consensus.cc:2258] T e0a99b19956b4bb8b366c05032e3f0c9 P ede3a6bd28b641ed99d02e817d287d20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:05.520488 5438 tablet_service.cc:1514] Processing DeleteTablet for tablet e0a99b19956b4bb8b366c05032e3f0c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:05 UTC) from {username='slave'} at 127.0.0.1:39994
I20260811 05:13:05.520671 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 5333
I20260811 05:13:05.523912 6117 tablet_replica.cc:330] T e0a99b19956b4bb8b366c05032e3f0c9 P cc0681329f1040af95adffe6df9b4b75: stopping tablet replica
I20260811 05:13:05.524062 6117 raft_consensus.cc:2229] T e0a99b19956b4bb8b366c05032e3f0c9 P cc0681329f1040af95adffe6df9b4b75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:05.524185 6117 raft_consensus.cc:2258] T e0a99b19956b4bb8b366c05032e3f0c9 P cc0681329f1040af95adffe6df9b4b75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:05.527357 6117 ts_tablet_manager.cc:1905] T e0a99b19956b4bb8b366c05032e3f0c9 P cc0681329f1040af95adffe6df9b4b75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:05.527691 6116 ts_tablet_manager.cc:1905] T e0a99b19956b4bb8b366c05032e3f0c9 P ede3a6bd28b641ed99d02e817d287d20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:05.529522 6116 ts_tablet_manager.cc:1918] T e0a99b19956b4bb8b366c05032e3f0c9 P ede3a6bd28b641ed99d02e817d287d20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:13:05.529539 6117 ts_tablet_manager.cc:1918] T e0a99b19956b4bb8b366c05032e3f0c9 P cc0681329f1040af95adffe6df9b4b75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:13:05.529613 6117 log.cc:1196] T e0a99b19956b4bb8b366c05032e3f0c9 P cc0681329f1040af95adffe6df9b4b75: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-2/wal/wals/e0a99b19956b4bb8b366c05032e3f0c9
I20260811 05:13:05.529614 6116 log.cc:1196] T e0a99b19956b4bb8b366c05032e3f0c9 P ede3a6bd28b641ed99d02e817d287d20: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425159060450-29246-0/minicluster-data/ts-1/wal/wals/e0a99b19956b4bb8b366c05032e3f0c9
I20260811 05:13:05.530048 6116 ts_tablet_manager.cc:1939] T e0a99b19956b4bb8b366c05032e3f0c9 P ede3a6bd28b641ed99d02e817d287d20: Deleting consensus metadata
I20260811 05:13:05.530048 6117 ts_tablet_manager.cc:1939] T e0a99b19956b4bb8b366c05032e3f0c9 P cc0681329f1040af95adffe6df9b4b75: Deleting consensus metadata
I20260811 05:13:05.530592 5192 catalog_manager.cc:4841] TS ede3a6bd28b641ed99d02e817d287d20 (127.28.143.130:34619): tablet e0a99b19956b4bb8b366c05032e3f0c9 (table test-table [id=f49591501e754583ae92b3a98235108c]) successfully deleted
I20260811 05:13:05.530687 5186 catalog_manager.cc:4841] TS cc0681329f1040af95adffe6df9b4b75 (127.28.143.131:46537): tablet e0a99b19956b4bb8b366c05032e3f0c9 (table test-table [id=f49591501e754583ae92b3a98235108c]) successfully deleted
W20260811 05:13:05.533032 5190 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.143.129:37163 (error 108)
I20260811 05:13:05.533106 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 5566
I20260811 05:13:05.538653 6066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:05.533432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:37163 (local address 127.0.0.1:40006)
0811 05:13:05.538497 (+ 5065us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:05.538605 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.129:37163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5023}
W20260811 05:13:05.540911 5190 catalog_manager.cc:4584] TS 513da2c177e348349c4801d8df3cbd7e (127.28.143.129:37163): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e0a99b19956b4bb8b366c05032e3f0c9: Network error: Client connection negotiation failed: client connection to 127.28.143.129:37163: connect: Connection refused (error 111)
I20260811 05:13:05.562479 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 5803
I20260811 05:13:05.593776 6066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:05.593580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:37163 (local address 127.0.0.1:40018)
0811 05:13:05.593667 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:05.593723 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.129:37163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 05:13:05.594230 6066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:05.594131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:37163 (local address 127.0.0.1:40020)
0811 05:13:05.594173 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:05.594205 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.129:37163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260811 05:13:05.596603 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 5162
2026-08-11T05:13:05Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (754 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 05:13:05.626574 29246 mini_oidc.cc:152] Starting JWKS server
I20260811 05:13:05.627904 29246 webserver.cc:460] Webserver started at https://127.0.0.1:43301/ using document root <none> and password file <none>
I20260811 05:13:05.628010 29246 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 05:13:05.628196 29246 webserver.cc:460] Webserver started at http://127.0.0.1:37599/ using document root <none> and password file <none>
2026-08-11T05:13:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:13:05Z Disabled control of system clock
I20260811 05:13:05.638044 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:36705
--webserver_interface=127.28.143.190
--webserver_port=0
--builtin_ntp_servers=127.28.143.148:41667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:36705
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43301/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786425159060450-29246-0/testchainca.cert with env {}
W20260811 05:13:05.730675 6136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:05.730911 6136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:05.730937 6136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:05.732204 6136 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:05.732249 6136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:05.732261 6136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:05.732273 6136 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:05.732285 6136 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 05:13:05.732295 6136 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 05:13:05.732306 6136 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43301/jwks.json
I20260811 05:13:05.734164 6136 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:41667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:36705
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:36705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43301/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786425159060450-29246-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:05.734406 6136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:05.734601 6136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:05.740263 6150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:05.740264 6147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:05.740263 6153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:05.740300 6136 server_base.cc:1034] running on GCE node
I20260811 05:13:05.741053 6136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:05.741379 6136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:05.742590 6136 hybrid_clock.cc:648] HybridClock initialized: now 1786425185742571 us; error 37 us; skew 500 ppm
I20260811 05:13:05.744117 6136 webserver.cc:460] Webserver started at http://127.28.143.190:40355/ using document root <none> and password file <none>
I20260811 05:13:05.744411 6136 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:05.744469 6136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:05.744608 6136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:05.745817 6136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "5e8abd7f150b4844a97bd77785fe122a"
format_stamp: "Formatted at 2026-08-11 05:13:05 on dist-test-slave-f7th"
server_key: "d005fcd2bcc69870e467c654bcc4a191"
server_key_iv: "6764850a0ba563a6b4a17c3e28b2424a"
server_key_version: "encryptionkey@0"
I20260811 05:13:05.746243 6136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "5e8abd7f150b4844a97bd77785fe122a"
format_stamp: "Formatted at 2026-08-11 05:13:05 on dist-test-slave-f7th"
server_key: "d005fcd2bcc69870e467c654bcc4a191"
server_key_iv: "6764850a0ba563a6b4a17c3e28b2424a"
server_key_version: "encryptionkey@0"
I20260811 05:13:05.747866 6136 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 05:13:05.748870 6164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:05.749089 6136 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 05:13:05.749200 6136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "5e8abd7f150b4844a97bd77785fe122a"
format_stamp: "Formatted at 2026-08-11 05:13:05 on dist-test-slave-f7th"
server_key: "d005fcd2bcc69870e467c654bcc4a191"
server_key_iv: "6764850a0ba563a6b4a17c3e28b2424a"
server_key_version: "encryptionkey@0"
I20260811 05:13:05.749275 6136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:05.758162 6136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:05.758597 6136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:05.758750 6136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:05.763211 6136 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:36705
I20260811 05:13:05.763255 6266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:36705 every 8 connection(s)
I20260811 05:13:05.763643 6136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:13:05.763808 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 6136
I20260811 05:13:05.763916 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:13:05.764118 29246 external_mini_cluster.cc:1428] Setting key fa2fd6f896ecb25ace4dec7e96ee8bbb
I20260811 05:13:05.765127 6267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:05.767939 6267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e8abd7f150b4844a97bd77785fe122a: Bootstrap starting.
I20260811 05:13:05.768610 6267 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e8abd7f150b4844a97bd77785fe122a: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:05.769292 6267 log.cc:824] T 00000000000000000000000000000000 P 5e8abd7f150b4844a97bd77785fe122a: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:05.770176 6272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:05.764709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53952 (local address 127.28.143.190:36705)
0811 05:13:05.765024 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:05.765033 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:05.765066 (+ 33us) server_negotiation.cc:407] Connection header received
0811 05:13:05.765580 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:05.765592 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:05.765666 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:05.765812 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:05.766039 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:05.766966 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:05.767768 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:05.768236 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:05.768357 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:05.769334 (+ 977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:05.769338 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:05.769338 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:13:05.769568 (+ 230us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:05.769617 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:13:05.769634 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:05.769710 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:05.769838 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 05:13:05.769941 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":105,"threads_started":1}
I20260811 05:13:05.770298 6267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e8abd7f150b4844a97bd77785fe122a: No bootstrap required, opened a new log
I20260811 05:13:05.771181 29246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:13:05.772877 6267 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e8abd7f150b4844a97bd77785fe122a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e8abd7f150b4844a97bd77785fe122a" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 36705 } }
I20260811 05:13:05.773041 6267 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e8abd7f150b4844a97bd77785fe122a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:05.773065 6267 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e8abd7f150b4844a97bd77785fe122a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e8abd7f150b4844a97bd77785fe122a, State: Initialized, Role: FOLLOWER
I20260811 05:13:05.773181 6267 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e8abd7f150b4844a97bd77785fe122a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e8abd7f150b4844a97bd77785fe122a" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 36705 } }
I20260811 05:13:05.773255 6267 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e8abd7f150b4844a97bd77785fe122a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:13:05.773283 6267 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e8abd7f150b4844a97bd77785fe122a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:13:05.773335 6267 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e8abd7f150b4844a97bd77785fe122a [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:05.773993 6267 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e8abd7f150b4844a97bd77785fe122a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e8abd7f150b4844a97bd77785fe122a" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 36705 } }
I20260811 05:13:05.774106 6267 leader_election.cc:302] T 00000000000000000000000000000000 P 5e8abd7f150b4844a97bd77785fe122a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e8abd7f150b4844a97bd77785fe122a; no voters:
I20260811 05:13:05.774279 6267 leader_election.cc:288] T 00000000000000000000000000000000 P 5e8abd7f150b4844a97bd77785fe122a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:13:05.774405 6277 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e8abd7f150b4844a97bd77785fe122a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:05.774572 6277 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e8abd7f150b4844a97bd77785fe122a [term 1 LEADER]: Becoming Leader. State: Replica: 5e8abd7f150b4844a97bd77785fe122a, State: Running, Role: LEADER
I20260811 05:13:05.774617 6267 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e8abd7f150b4844a97bd77785fe122a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:13:05.774708 6277 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e8abd7f150b4844a97bd77785fe122a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e8abd7f150b4844a97bd77785fe122a" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 36705 } }
I20260811 05:13:05.775125 6278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e8abd7f150b4844a97bd77785fe122a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5e8abd7f150b4844a97bd77785fe122a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e8abd7f150b4844a97bd77785fe122a" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 36705 } } }
I20260811 05:13:05.775132 6279 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e8abd7f150b4844a97bd77785fe122a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e8abd7f150b4844a97bd77785fe122a. Latest consensus state: current_term: 1 leader_uuid: "5e8abd7f150b4844a97bd77785fe122a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e8abd7f150b4844a97bd77785fe122a" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 36705 } } }
I20260811 05:13:05.775276 6278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e8abd7f150b4844a97bd77785fe122a [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:05.775295 6279 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e8abd7f150b4844a97bd77785fe122a [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:05.775589 6293 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:13:05.776268 6293 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:13:05.777863 6293 catalog_manager.cc:1347] Generated new cluster ID: 3fa8d85c8b3246d7b4366c9c2a7eaee7
I20260811 05:13:05.777922 6293 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:13:05.784600 6293 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:13:05.785187 6293 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:13:05.787324 6293 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e8abd7f150b4844a97bd77785fe122a: Generated new TSK 0
I20260811 05:13:05.787519 6293 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 05:13:05.817404 6320 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 05:13:05.817646 6320 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 05:13:05.817691 6272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:05.794099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53956 (local address 127.28.143.190:36705)
0811 05:13:05.794150 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:05.794152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:05.794283 (+ 131us) server_negotiation.cc:407] Connection header received
0811 05:13:05.794395 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:05.794396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:05.794408 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:05.794466 (+ 58us) server_negotiation.cc:226] Negotiated authn=JWT
0811 05:13:05.794617 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:05.795384 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:05.796189 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:05.796411 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:05.796467 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:05.796725 (+ 258us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 05:13:05.817353 (+ 20628us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43301/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 05:13:05.817614 (+ 261us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53956: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43301/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":27}
W20260811 05:13:05.817785 6272 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53956: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43301/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 05:13:05.818102 6319 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:05.794007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36705 (local address 127.0.0.1:53956)
0811 05:13:05.794213 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:05.794224 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:05.794296 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:05.794480 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:05.794481 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:05.794483 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0811 05:13:05.794555 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:05.794558 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:05.795490 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:05.795491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:05.796106 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:05.796109 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:05.796488 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:05.796489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:05.796551 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:05.796577 (+ 26us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 05:13:05.817525 (+ 20948us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43301/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 05:13:05.817661 (+ 136us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.28.143.190:36705: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43301/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":102,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.28.143.190:36705: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43301/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 05:13:05.818722 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 6136
2026-08-11T05:13:05Z chronyd exiting
I20260811 05:13:05.872273 29246 test_util.cc:175] -----------------------------------------------
I20260811 05:13:05.872381 29246 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786425159060450-29246-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (247 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 05:13:05.873328 29246 mini_oidc.cc:152] Starting JWKS server
I20260811 05:13:05.874158 29246 webserver.cc:460] Webserver started at https://127.0.0.1:40853/ using document root <none> and password file <none>
I20260811 05:13:05.874228 29246 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 05:13:05.874405 29246 webserver.cc:460] Webserver started at http://127.0.0.1:38027/ using document root <none> and password file <none>
2026-08-11T05:13:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:13:05Z Disabled control of system clock
I20260811 05:13:05.885175 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:44777
--webserver_interface=127.28.143.190
--webserver_port=0
--builtin_ntp_servers=127.28.143.148:42669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:44777
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40853/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786425159060450-29246-0/testchainca.cert with env {}
W20260811 05:13:05.975469 6328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:05.975714 6328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:05.975739 6328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:05.977092 6328 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:05.977151 6328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:05.977164 6328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:05.977175 6328 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:05.977186 6328 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 05:13:05.977197 6328 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 05:13:05.977207 6328 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40853/jwks.json
I20260811 05:13:05.979009 6328 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:42669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:44777
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:44777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40853/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786425159060450-29246-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:05.979275 6328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:05.979503 6328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:05.984691 6338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:05.984735 6341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:05.984740 6337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:05.985317 6328 server_base.cc:1034] running on GCE node
I20260811 05:13:05.985613 6328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:05.985955 6328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:05.987145 6328 hybrid_clock.cc:648] HybridClock initialized: now 1786425185987123 us; error 35 us; skew 500 ppm
I20260811 05:13:05.988557 6328 webserver.cc:460] Webserver started at http://127.28.143.190:41425/ using document root <none> and password file <none>
I20260811 05:13:05.988839 6328 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:05.988895 6328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:05.989032 6328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:05.990135 6328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "0a26ec5502864421a5d0cc745a1b6966"
format_stamp: "Formatted at 2026-08-11 05:13:05 on dist-test-slave-f7th"
server_key: "e2bb362afb864cb81a4c4f9dfed0af80"
server_key_iv: "9d0f5411f37261fe388b1d79c39183d6"
server_key_version: "encryptionkey@0"
I20260811 05:13:05.990492 6328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "0a26ec5502864421a5d0cc745a1b6966"
format_stamp: "Formatted at 2026-08-11 05:13:05 on dist-test-slave-f7th"
server_key: "e2bb362afb864cb81a4c4f9dfed0af80"
server_key_iv: "9d0f5411f37261fe388b1d79c39183d6"
server_key_version: "encryptionkey@0"
I20260811 05:13:05.992180 6328 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 05:13:05.993254 6355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:05.993553 6328 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 05:13:05.993667 6328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "0a26ec5502864421a5d0cc745a1b6966"
format_stamp: "Formatted at 2026-08-11 05:13:05 on dist-test-slave-f7th"
server_key: "e2bb362afb864cb81a4c4f9dfed0af80"
server_key_iv: "9d0f5411f37261fe388b1d79c39183d6"
server_key_version: "encryptionkey@0"
I20260811 05:13:05.993745 6328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:06.005985 6328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:06.006343 6328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:06.006491 6328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:06.011577 6328 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:44777
I20260811 05:13:06.011613 6447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:44777 every 8 connection(s)
I20260811 05:13:06.012043 6328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:13:06.012965 6448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:06.015969 6448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a26ec5502864421a5d0cc745a1b6966: Bootstrap starting.
I20260811 05:13:06.016600 6448 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a26ec5502864421a5d0cc745a1b6966: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:06.016947 6448 log.cc:824] T 00000000000000000000000000000000 P 0a26ec5502864421a5d0cc745a1b6966: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:06.017663 6448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a26ec5502864421a5d0cc745a1b6966: No bootstrap required, opened a new log
I20260811 05:13:06.019345 6448 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a26ec5502864421a5d0cc745a1b6966 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a26ec5502864421a5d0cc745a1b6966" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 44777 } }
I20260811 05:13:06.019618 6448 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a26ec5502864421a5d0cc745a1b6966 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:06.019661 6448 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a26ec5502864421a5d0cc745a1b6966 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a26ec5502864421a5d0cc745a1b6966, State: Initialized, Role: FOLLOWER
I20260811 05:13:06.019843 6448 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a26ec5502864421a5d0cc745a1b6966 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a26ec5502864421a5d0cc745a1b6966" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 44777 } }
I20260811 05:13:06.019932 6448 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a26ec5502864421a5d0cc745a1b6966 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:13:06.019958 6448 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a26ec5502864421a5d0cc745a1b6966 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:13:06.019987 6448 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a26ec5502864421a5d0cc745a1b6966 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:06.020596 6448 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a26ec5502864421a5d0cc745a1b6966 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a26ec5502864421a5d0cc745a1b6966" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 44777 } }
I20260811 05:13:06.020742 6448 leader_election.cc:302] T 00000000000000000000000000000000 P 0a26ec5502864421a5d0cc745a1b6966 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a26ec5502864421a5d0cc745a1b6966; no voters:
I20260811 05:13:06.020893 6448 leader_election.cc:288] T 00000000000000000000000000000000 P 0a26ec5502864421a5d0cc745a1b6966 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:13:06.020972 6457 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a26ec5502864421a5d0cc745a1b6966 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:06.021142 6457 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a26ec5502864421a5d0cc745a1b6966 [term 1 LEADER]: Becoming Leader. State: Replica: 0a26ec5502864421a5d0cc745a1b6966, State: Running, Role: LEADER
I20260811 05:13:06.021221 6448 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a26ec5502864421a5d0cc745a1b6966 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:13:06.021250 6457 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a26ec5502864421a5d0cc745a1b6966 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a26ec5502864421a5d0cc745a1b6966" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 44777 } }
I20260811 05:13:06.021744 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 6328
I20260811 05:13:06.021849 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:13:06.022042 29246 external_mini_cluster.cc:1428] Setting key c8911c00d1ac6692306665b7d4fa85aa
I20260811 05:13:06.022596 6458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a26ec5502864421a5d0cc745a1b6966 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a26ec5502864421a5d0cc745a1b6966" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a26ec5502864421a5d0cc745a1b6966" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 44777 } } }
I20260811 05:13:06.022699 6458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a26ec5502864421a5d0cc745a1b6966 [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:06.022574 6459 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a26ec5502864421a5d0cc745a1b6966 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a26ec5502864421a5d0cc745a1b6966. Latest consensus state: current_term: 1 leader_uuid: "0a26ec5502864421a5d0cc745a1b6966" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a26ec5502864421a5d0cc745a1b6966" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 44777 } } }
I20260811 05:13:06.022887 6459 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a26ec5502864421a5d0cc745a1b6966 [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:06.022933 6476 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:13:06.023459 6476 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:13:06.025221 6476 catalog_manager.cc:1347] Generated new cluster ID: 2671a8ef9df2489cbd70d84c24ccfdb4
I20260811 05:13:06.025285 6476 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:13:06.028803 6474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:06.022544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50516 (local address 127.28.143.190:44777)
0811 05:13:06.022777 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:06.022787 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:06.022824 (+ 37us) server_negotiation.cc:407] Connection header received
0811 05:13:06.023076 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:06.023084 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:06.023161 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:06.023285 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:06.023698 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:06.024951 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:06.026152 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:06.026831 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:06.027029 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:06.027804 (+ 775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:06.027808 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:06.027809 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:13:06.028153 (+ 344us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:06.028201 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:13:06.028219 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:06.028296 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:06.028413 (+ 117us) server_negotiation.cc:299] Negotiation successful
0811 05:13:06.028515 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":73,"threads_started":1}
I20260811 05:13:06.029788 29246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:13:06.047241 6476 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:13:06.048027 6476 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:13:06.061764 6476 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a26ec5502864421a5d0cc745a1b6966: Generated new TSK 0
I20260811 05:13:06.062031 6476 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 05:13:06.083998 6502 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 05:13:06.084106 6502 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 05:13:06.084414 6474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:06.069595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50518 (local address 127.28.143.190:44777)
0811 05:13:06.069654 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:06.069656 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:06.069666 (+ 10us) server_negotiation.cc:407] Connection header received
0811 05:13:06.069710 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:06.069710 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:06.069722 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:06.069792 (+ 70us) server_negotiation.cc:226] Negotiated authn=JWT
0811 05:13:06.069950 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:06.070464 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:06.071633 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:06.071876 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:06.071944 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:06.072190 (+ 246us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 05:13:06.084204 (+ 12014us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40853/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 05:13:06.084346 (+ 142us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50518: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40853/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":34}
W20260811 05:13:06.084463 6474 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50518: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40853/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260811 05:13:06.084599 6499 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:06.068772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:44777 (local address 127.0.0.1:50518)
0811 05:13:06.068950 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:06.068963 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:06.069084 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:06.069811 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:06.069812 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:06.069815 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0811 05:13:06.069881 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:06.069883 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:06.070541 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:06.070542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:06.071500 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:06.071504 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:06.071964 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:06.071965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:06.072032 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:06.072040 (+ 8us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 05:13:06.084342 (+ 12302us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40853/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 05:13:06.084490 (+ 148us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.28.143.190:44777: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40853/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":84,"threads_started":1}
I20260811 05:13:06.084908 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 6328
2026-08-11T05:13:06Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (262 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 05:13:06.135847 29246 mini_oidc.cc:152] Starting JWKS server
I20260811 05:13:06.136551 29246 webserver.cc:460] Webserver started at https://127.0.0.1:40003/ using document root <none> and password file <none>
I20260811 05:13:06.136600 29246 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 05:13:06.136770 29246 webserver.cc:460] Webserver started at http://127.0.0.1:32773/ using document root <none> and password file <none>
2026-08-11T05:13:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:13:06Z Disabled control of system clock
I20260811 05:13:06.147718 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:44089
--webserver_interface=127.28.143.190
--webserver_port=0
--builtin_ntp_servers=127.28.143.148:45021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:44089
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40003/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 05:13:06.234593 6515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:06.234848 6515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:06.234884 6515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:06.236515 6515 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:06.236584 6515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:06.236603 6515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:06.236647 6515 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:06.236665 6515 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 05:13:06.236683 6515 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 05:13:06.236706 6515 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40003/jwks.json
I20260811 05:13:06.238464 6515 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:45021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:44089
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:44089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40003/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:06.238790 6515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:06.239120 6515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:06.244495 6540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:06.244534 6527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:06.244581 6537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:06.244760 6515 server_base.cc:1034] running on GCE node
I20260811 05:13:06.245199 6515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:06.245512 6515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:06.246682 6515 hybrid_clock.cc:648] HybridClock initialized: now 1786425186246657 us; error 39 us; skew 500 ppm
I20260811 05:13:06.248112 6515 webserver.cc:460] Webserver started at http://127.28.143.190:35099/ using document root <none> and password file <none>
I20260811 05:13:06.248361 6515 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:06.248410 6515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:06.248556 6515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:06.249729 6515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "6c57df82bcd5450d932b58c13d58f59b"
format_stamp: "Formatted at 2026-08-11 05:13:06 on dist-test-slave-f7th"
server_key: "70d65bdc2ca4e6729dffa8a849ab68bb"
server_key_iv: "baa10cc3c6727396c7d1dce06695351f"
server_key_version: "encryptionkey@0"
I20260811 05:13:06.250058 6515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "6c57df82bcd5450d932b58c13d58f59b"
format_stamp: "Formatted at 2026-08-11 05:13:06 on dist-test-slave-f7th"
server_key: "70d65bdc2ca4e6729dffa8a849ab68bb"
server_key_iv: "baa10cc3c6727396c7d1dce06695351f"
server_key_version: "encryptionkey@0"
I20260811 05:13:06.251494 6515 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 05:13:06.252928 6557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:06.253162 6515 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:06.253269 6515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "6c57df82bcd5450d932b58c13d58f59b"
format_stamp: "Formatted at 2026-08-11 05:13:06 on dist-test-slave-f7th"
server_key: "70d65bdc2ca4e6729dffa8a849ab68bb"
server_key_iv: "baa10cc3c6727396c7d1dce06695351f"
server_key_version: "encryptionkey@0"
I20260811 05:13:06.253352 6515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:06.265089 6515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:06.265450 6515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:06.265588 6515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:06.269814 6515 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:44089
I20260811 05:13:06.269855 6644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:44089 every 8 connection(s)
I20260811 05:13:06.270203 6515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:13:06.270905 6646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:06.273962 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 6515
I20260811 05:13:06.274086 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:13:06.274258 29246 external_mini_cluster.cc:1428] Setting key 5afc71f6068ecc58b7d5828263814291
I20260811 05:13:06.274330 6646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c57df82bcd5450d932b58c13d58f59b: Bootstrap starting.
I20260811 05:13:06.275285 6646 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c57df82bcd5450d932b58c13d58f59b: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:06.275620 6646 log.cc:824] T 00000000000000000000000000000000 P 6c57df82bcd5450d932b58c13d58f59b: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:06.276466 6646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c57df82bcd5450d932b58c13d58f59b: No bootstrap required, opened a new log
I20260811 05:13:06.278338 6646 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c57df82bcd5450d932b58c13d58f59b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c57df82bcd5450d932b58c13d58f59b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 44089 } }
I20260811 05:13:06.278501 6646 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c57df82bcd5450d932b58c13d58f59b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:06.278535 6646 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c57df82bcd5450d932b58c13d58f59b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c57df82bcd5450d932b58c13d58f59b, State: Initialized, Role: FOLLOWER
I20260811 05:13:06.278667 6646 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c57df82bcd5450d932b58c13d58f59b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c57df82bcd5450d932b58c13d58f59b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 44089 } }
I20260811 05:13:06.278735 6646 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c57df82bcd5450d932b58c13d58f59b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:13:06.278766 6646 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c57df82bcd5450d932b58c13d58f59b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:13:06.278815 6646 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c57df82bcd5450d932b58c13d58f59b [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:06.279423 6646 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c57df82bcd5450d932b58c13d58f59b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c57df82bcd5450d932b58c13d58f59b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 44089 } }
I20260811 05:13:06.279527 6646 leader_election.cc:302] T 00000000000000000000000000000000 P 6c57df82bcd5450d932b58c13d58f59b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c57df82bcd5450d932b58c13d58f59b; no voters:
I20260811 05:13:06.279706 6646 leader_election.cc:288] T 00000000000000000000000000000000 P 6c57df82bcd5450d932b58c13d58f59b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:13:06.279805 6653 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c57df82bcd5450d932b58c13d58f59b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:06.279978 6653 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c57df82bcd5450d932b58c13d58f59b [term 1 LEADER]: Becoming Leader. State: Replica: 6c57df82bcd5450d932b58c13d58f59b, State: Running, Role: LEADER
I20260811 05:13:06.279980 6646 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c57df82bcd5450d932b58c13d58f59b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:13:06.280112 6653 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c57df82bcd5450d932b58c13d58f59b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c57df82bcd5450d932b58c13d58f59b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 44089 } }
I20260811 05:13:06.280524 6654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c57df82bcd5450d932b58c13d58f59b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c57df82bcd5450d932b58c13d58f59b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c57df82bcd5450d932b58c13d58f59b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 44089 } } }
I20260811 05:13:06.280535 6655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c57df82bcd5450d932b58c13d58f59b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c57df82bcd5450d932b58c13d58f59b. Latest consensus state: current_term: 1 leader_uuid: "6c57df82bcd5450d932b58c13d58f59b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c57df82bcd5450d932b58c13d58f59b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 44089 } } }
I20260811 05:13:06.280717 6654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c57df82bcd5450d932b58c13d58f59b [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:06.280735 6655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c57df82bcd5450d932b58c13d58f59b [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:06.280927 6651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:06.274984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41556 (local address 127.28.143.190:44089)
0811 05:13:06.275219 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:06.275226 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:06.275253 (+ 27us) server_negotiation.cc:407] Connection header received
0811 05:13:06.275923 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:06.275932 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:06.276016 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:06.276305 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:06.276564 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:06.277735 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:06.278569 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:06.279089 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:06.279205 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:06.279936 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:06.279940 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:06.279941 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:13:06.280163 (+ 222us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:06.280208 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:13:06.280224 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:06.280284 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:06.280513 (+ 229us) server_negotiation.cc:299] Negotiation successful
0811 05:13:06.280608 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":112,"threads_started":1}
I20260811 05:13:06.280941 6657 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:13:06.281481 6657 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:13:06.282018 29246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:13:06.283257 6657 catalog_manager.cc:1347] Generated new cluster ID: 68a82563ae8e4f7fb2528df529932ed9
I20260811 05:13:06.283310 6657 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:13:06.294142 6657 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:13:06.294672 6657 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:13:06.298262 6657 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c57df82bcd5450d932b58c13d58f59b: Generated new TSK 0
I20260811 05:13:06.298448 6657 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:13:06.317173 6651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:06.312944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41562 (local address 127.28.143.190:44089)
0811 05:13:06.313005 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:06.313007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:06.313097 (+ 90us) server_negotiation.cc:407] Connection header received
0811 05:13:06.313160 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:06.313161 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:06.313174 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:06.313239 (+ 65us) server_negotiation.cc:226] Negotiated authn=JWT
0811 05:13:06.313473 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:06.314030 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:06.315309 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:06.315530 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:06.315593 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:06.315878 (+ 285us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 05:13:06.316857 (+ 979us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40003/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 05:13:06.317005 (+ 148us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41562: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40003/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":35}
W20260811 05:13:06.317256 6693 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:06.312828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:44089 (local address 127.0.0.1:41562)
0811 05:13:06.312999 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:06.313011 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:06.313086 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:06.313254 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:06.313255 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:06.313257 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0811 05:13:06.313323 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:06.313326 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:06.314113 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:06.314114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:06.315186 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:06.315191 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:06.315645 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:06.315647 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:06.315721 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:06.315729 (+ 8us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 05:13:06.316961 (+ 1232us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40003/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 05:13:06.317153 (+ 192us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.28.143.190:44089: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40003/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":99,"threads_started":1}
W20260811 05:13:06.317265 6651 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41562: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40003/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
I20260811 05:13:06.317612 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 6515
E20260811 05:13:06.375010 6695 webserver.cc:533] Webserver: sslize failed:
E20260811 05:13:06.375342 6695 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-11T05:13:06Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (271 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T05:13:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:13:06Z Disabled control of system clock
I20260811 05:13:06.416838 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:42987
--webserver_interface=127.28.143.190
--webserver_port=0
--builtin_ntp_servers=127.28.143.148:37009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:42987
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786425159060450-29246-0/testchainca.cert with env {}
W20260811 05:13:06.504985 6703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:06.505235 6703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:06.505260 6703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:06.506727 6703 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:06.506958 6703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:06.507145 6703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:06.507167 6703 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:06.507180 6703 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 05:13:06.507193 6703 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 05:13:06.507203 6703 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 05:13:06.509286 6703 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:37009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:42987
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:42987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786425159060450-29246-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:06.509591 6703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:06.509821 6703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:06.515375 6719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:06.515393 6713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:06.515431 6703 server_base.cc:1034] running on GCE node
W20260811 05:13:06.515377 6724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:06.515928 6703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:06.516284 6703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:06.517477 6703 hybrid_clock.cc:648] HybridClock initialized: now 1786425186517446 us; error 48 us; skew 500 ppm
I20260811 05:13:06.518954 6703 webserver.cc:460] Webserver started at http://127.28.143.190:40431/ using document root <none> and password file <none>
I20260811 05:13:06.519335 6703 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:06.519526 6703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:06.519790 6703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:06.521059 6703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "bac28e12bb104830bcf02ee95c0ee6cf"
format_stamp: "Formatted at 2026-08-11 05:13:06 on dist-test-slave-f7th"
server_key: "7e2a5fe4740cd853c1fcc2060c5ec35e"
server_key_iv: "9020a4c3ab98150c15ebfdfc5ea430e6"
server_key_version: "encryptionkey@0"
I20260811 05:13:06.521426 6703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "bac28e12bb104830bcf02ee95c0ee6cf"
format_stamp: "Formatted at 2026-08-11 05:13:06 on dist-test-slave-f7th"
server_key: "7e2a5fe4740cd853c1fcc2060c5ec35e"
server_key_iv: "9020a4c3ab98150c15ebfdfc5ea430e6"
server_key_version: "encryptionkey@0"
I20260811 05:13:06.522943 6703 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 05:13:06.523960 6732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:06.524243 6703 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:06.524355 6703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "bac28e12bb104830bcf02ee95c0ee6cf"
format_stamp: "Formatted at 2026-08-11 05:13:06 on dist-test-slave-f7th"
server_key: "7e2a5fe4740cd853c1fcc2060c5ec35e"
server_key_iv: "9020a4c3ab98150c15ebfdfc5ea430e6"
server_key_version: "encryptionkey@0"
I20260811 05:13:06.524443 6703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:06.539566 6703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:06.539943 6703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:06.540086 6703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:06.544368 6703 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:42987
I20260811 05:13:06.544409 6834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:42987 every 8 connection(s)
I20260811 05:13:06.544806 6703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:13:06.545913 6838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:06.549216 6838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bac28e12bb104830bcf02ee95c0ee6cf: Bootstrap starting.
I20260811 05:13:06.550061 6838 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bac28e12bb104830bcf02ee95c0ee6cf: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:06.550338 6838 log.cc:824] T 00000000000000000000000000000000 P bac28e12bb104830bcf02ee95c0ee6cf: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:06.551088 6838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bac28e12bb104830bcf02ee95c0ee6cf: No bootstrap required, opened a new log
I20260811 05:13:06.552983 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 6703
I20260811 05:13:06.553088 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:13:06.553282 29246 external_mini_cluster.cc:1428] Setting key 540075ce5e26f279ebd6e82c2674e974
I20260811 05:13:06.553203 6838 raft_consensus.cc:348] T 00000000000000000000000000000000 P bac28e12bb104830bcf02ee95c0ee6cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bac28e12bb104830bcf02ee95c0ee6cf" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 42987 } }
I20260811 05:13:06.553406 6838 raft_consensus.cc:374] T 00000000000000000000000000000000 P bac28e12bb104830bcf02ee95c0ee6cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:06.553450 6838 raft_consensus.cc:729] T 00000000000000000000000000000000 P bac28e12bb104830bcf02ee95c0ee6cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bac28e12bb104830bcf02ee95c0ee6cf, State: Initialized, Role: FOLLOWER
I20260811 05:13:06.553593 6838 consensus_queue.cc:260] T 00000000000000000000000000000000 P bac28e12bb104830bcf02ee95c0ee6cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bac28e12bb104830bcf02ee95c0ee6cf" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 42987 } }
I20260811 05:13:06.553673 6838 raft_consensus.cc:388] T 00000000000000000000000000000000 P bac28e12bb104830bcf02ee95c0ee6cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:13:06.553697 6838 raft_consensus.cc:482] T 00000000000000000000000000000000 P bac28e12bb104830bcf02ee95c0ee6cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:13:06.553738 6838 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bac28e12bb104830bcf02ee95c0ee6cf [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:06.554543 6838 raft_consensus.cc:504] T 00000000000000000000000000000000 P bac28e12bb104830bcf02ee95c0ee6cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bac28e12bb104830bcf02ee95c0ee6cf" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 42987 } }
I20260811 05:13:06.554670 6838 leader_election.cc:302] T 00000000000000000000000000000000 P bac28e12bb104830bcf02ee95c0ee6cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bac28e12bb104830bcf02ee95c0ee6cf; no voters:
I20260811 05:13:06.554842 6838 leader_election.cc:288] T 00000000000000000000000000000000 P bac28e12bb104830bcf02ee95c0ee6cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:13:06.555114 6845 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bac28e12bb104830bcf02ee95c0ee6cf [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:06.555120 6838 sys_catalog.cc:564] T 00000000000000000000000000000000 P bac28e12bb104830bcf02ee95c0ee6cf [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:13:06.555284 6845 raft_consensus.cc:686] T 00000000000000000000000000000000 P bac28e12bb104830bcf02ee95c0ee6cf [term 1 LEADER]: Becoming Leader. State: Replica: bac28e12bb104830bcf02ee95c0ee6cf, State: Running, Role: LEADER
I20260811 05:13:06.555385 6845 consensus_queue.cc:237] T 00000000000000000000000000000000 P bac28e12bb104830bcf02ee95c0ee6cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bac28e12bb104830bcf02ee95c0ee6cf" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 42987 } }
I20260811 05:13:06.555747 6846 sys_catalog.cc:455] T 00000000000000000000000000000000 P bac28e12bb104830bcf02ee95c0ee6cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bac28e12bb104830bcf02ee95c0ee6cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bac28e12bb104830bcf02ee95c0ee6cf" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 42987 } } }
I20260811 05:13:06.555842 6846 sys_catalog.cc:458] T 00000000000000000000000000000000 P bac28e12bb104830bcf02ee95c0ee6cf [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:06.555751 6847 sys_catalog.cc:455] T 00000000000000000000000000000000 P bac28e12bb104830bcf02ee95c0ee6cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader bac28e12bb104830bcf02ee95c0ee6cf. Latest consensus state: current_term: 1 leader_uuid: "bac28e12bb104830bcf02ee95c0ee6cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bac28e12bb104830bcf02ee95c0ee6cf" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 42987 } } }
I20260811 05:13:06.555927 6847 sys_catalog.cc:458] T 00000000000000000000000000000000 P bac28e12bb104830bcf02ee95c0ee6cf [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:06.556109 6861 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:13:06.556773 6861 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:13:06.558693 6861 catalog_manager.cc:1347] Generated new cluster ID: 14b02c83381846dc9a5c34a507d282d0
I20260811 05:13:06.558758 6861 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:13:06.559442 6844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:06.553794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41124 (local address 127.28.143.190:42987)
0811 05:13:06.554027 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:06.554036 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:06.554071 (+ 35us) server_negotiation.cc:407] Connection header received
0811 05:13:06.554204 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:06.554212 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:06.554301 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:06.554476 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:06.554726 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:06.556233 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:06.557394 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:06.557879 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:06.558009 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:06.558721 (+ 712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:06.558725 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:06.558725 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:13:06.558932 (+ 207us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:06.558974 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:13:06.558988 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:06.559048 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:06.559158 (+ 110us) server_negotiation.cc:299] Negotiation successful
0811 05:13:06.559233 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":87,"threads_started":1}
I20260811 05:13:06.567065 29246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:13:06.576721 6861 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:13:06.577275 6861 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:13:06.581530 6861 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bac28e12bb104830bcf02ee95c0ee6cf: Generated new TSK 0
I20260811 05:13:06.581743 6861 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:13:08.008351 6844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:06.589655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41126 (local address 127.28.143.190:42987)
0811 05:13:06.589708 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:06.589710 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:06.589792 (+ 82us) server_negotiation.cc:407] Connection header received
0811 05:13:06.589899 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:06.589900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:06.589911 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:06.589974 (+ 63us) server_negotiation.cc:226] Negotiated authn=JWT
0811 05:13:06.590120 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:06.590631 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:06.591797 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:06.592026 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:06.592091 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:06.592371 (+ 280us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 05:13:08.008105 (+1415734us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 05:13:08.008283 (+ 178us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41126: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":28}
W20260811 05:13:08.008440 6844 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41126: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260811 05:13:08.008541 6889 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:06.589567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:42987 (local address 127.0.0.1:41126)
0811 05:13:06.589721 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:06.589733 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:06.589802 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:06.589983 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:06.589984 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:06.589986 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0811 05:13:06.590058 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:06.590060 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:06.590700 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:06.590701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:06.591670 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:06.591674 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:06.592107 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:06.592108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:06.592171 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:06.592178 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 05:13:08.008268 (+1416090us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 05:13:08.008404 (+ 136us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.28.143.190:42987: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":87,"threads_started":1}
I20260811 05:13:08.008915 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 6703
2026-08-11T05:13:08Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1629 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (95 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (95 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:13:08 dist-test-slave-f7th krb5kdc[6910](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:13:08 dist-test-slave-f7th krb5kdc[6910](info): set up 2 sockets
Aug 11 05:13:08 dist-test-slave-f7th krb5kdc[6910](info): commencing operation
krb5kdc: starting...
W20260811 05:13:09.443269 29246 mini_kdc.cc:121] Time spent starting KDC: real 1.215s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Aug 11 05:13:09 dist-test-slave-f7th krb5kdc[6910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425189, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:13:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:13:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
I20260811 05:13:09.580657 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:42997
--webserver_interface=127.28.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:38267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:42997
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:09.667030 6936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:09.667263 6936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:09.667289 6936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:09.668609 6936 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:09.668691 6936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:09.668704 6936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:09.668716 6936 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:09.668726 6936 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:13:09.670312 6936 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:38267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:42997
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:42997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:09.670583 6936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:09.670815 6936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:09.675907 6956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:09.675920 6946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:09.675911 6950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:09.676220 6936 server_base.cc:1034] running on GCE node
I20260811 05:13:09.676550 6936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:09.676903 6936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:09.678081 6936 hybrid_clock.cc:648] HybridClock initialized: now 1786425189678062 us; error 40 us; skew 500 ppm
Aug 11 05:13:09 dist-test-slave-f7th krb5kdc[6910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425189, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:09.680598 6936 init.cc:377] Logged in from keytab as kudu/127.28.143.190@KRBTEST.COM (short username kudu)
I20260811 05:13:09.681099 6936 webserver.cc:460] Webserver started at http://127.28.143.190:43943/ using document root <none> and password file <none>
I20260811 05:13:09.681315 6936 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:09.681361 6936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:09.681492 6936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:09.682632 6936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "0580d2ae5a674b82993e5dae8978293c"
format_stamp: "Formatted at 2026-08-11 05:13:09 on dist-test-slave-f7th"
server_key: "95284828b304528ffa73ac5ab5559858"
server_key_iv: "073a7c745d83caad437843c152e304a8"
server_key_version: "encryptionkey@0"
I20260811 05:13:09.682973 6936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "0580d2ae5a674b82993e5dae8978293c"
format_stamp: "Formatted at 2026-08-11 05:13:09 on dist-test-slave-f7th"
server_key: "95284828b304528ffa73ac5ab5559858"
server_key_iv: "073a7c745d83caad437843c152e304a8"
server_key_version: "encryptionkey@0"
I20260811 05:13:09.684422 6936 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260811 05:13:09.685398 6976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:09.685593 6936 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 05:13:09.685695 6936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "0580d2ae5a674b82993e5dae8978293c"
format_stamp: "Formatted at 2026-08-11 05:13:09 on dist-test-slave-f7th"
server_key: "95284828b304528ffa73ac5ab5559858"
server_key_iv: "073a7c745d83caad437843c152e304a8"
server_key_version: "encryptionkey@0"
I20260811 05:13:09.685758 6936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:09.700232 6936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:09.702533 6936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:09.702665 6936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:09.706656 6936 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:42997
I20260811 05:13:09.706693 7071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:42997 every 8 connection(s)
I20260811 05:13:09.707074 6936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:13:09.707983 7075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:09.710315 7075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0580d2ae5a674b82993e5dae8978293c: Bootstrap starting.
I20260811 05:13:09.710907 7075 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0580d2ae5a674b82993e5dae8978293c: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:09.711190 7075 log.cc:824] T 00000000000000000000000000000000 P 0580d2ae5a674b82993e5dae8978293c: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:09.711889 7075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0580d2ae5a674b82993e5dae8978293c: No bootstrap required, opened a new log
I20260811 05:13:09.713587 7075 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0580d2ae5a674b82993e5dae8978293c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0580d2ae5a674b82993e5dae8978293c" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 42997 } }
I20260811 05:13:09.713732 7075 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0580d2ae5a674b82993e5dae8978293c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:09.713766 7075 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0580d2ae5a674b82993e5dae8978293c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0580d2ae5a674b82993e5dae8978293c, State: Initialized, Role: FOLLOWER
I20260811 05:13:09.713910 7075 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0580d2ae5a674b82993e5dae8978293c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0580d2ae5a674b82993e5dae8978293c" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 42997 } }
I20260811 05:13:09.713979 7075 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0580d2ae5a674b82993e5dae8978293c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:13:09.714011 7075 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0580d2ae5a674b82993e5dae8978293c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:13:09.714063 7075 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0580d2ae5a674b82993e5dae8978293c [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:09.714636 7075 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0580d2ae5a674b82993e5dae8978293c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0580d2ae5a674b82993e5dae8978293c" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 42997 } }
I20260811 05:13:09.714793 7075 leader_election.cc:302] T 00000000000000000000000000000000 P 0580d2ae5a674b82993e5dae8978293c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0580d2ae5a674b82993e5dae8978293c; no voters:
I20260811 05:13:09.715066 7075 leader_election.cc:288] T 00000000000000000000000000000000 P 0580d2ae5a674b82993e5dae8978293c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:13:09.715153 7079 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0580d2ae5a674b82993e5dae8978293c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:09.715337 7075 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0580d2ae5a674b82993e5dae8978293c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:13:09.715524 7079 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0580d2ae5a674b82993e5dae8978293c [term 1 LEADER]: Becoming Leader. State: Replica: 0580d2ae5a674b82993e5dae8978293c, State: Running, Role: LEADER
I20260811 05:13:09.715659 7079 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0580d2ae5a674b82993e5dae8978293c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0580d2ae5a674b82993e5dae8978293c" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 42997 } }
I20260811 05:13:09.716114 7081 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0580d2ae5a674b82993e5dae8978293c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0580d2ae5a674b82993e5dae8978293c. Latest consensus state: current_term: 1 leader_uuid: "0580d2ae5a674b82993e5dae8978293c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0580d2ae5a674b82993e5dae8978293c" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 42997 } } }
I20260811 05:13:09.716051 7080 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0580d2ae5a674b82993e5dae8978293c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0580d2ae5a674b82993e5dae8978293c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0580d2ae5a674b82993e5dae8978293c" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 42997 } } }
I20260811 05:13:09.716248 7081 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0580d2ae5a674b82993e5dae8978293c [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:09.716262 7080 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0580d2ae5a674b82993e5dae8978293c [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:09.716509 7095 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:13:09.717127 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 6936
I20260811 05:13:09.717252 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:13:09.717473 29246 external_mini_cluster.cc:1428] Setting key bf026202992e78a5d05986709f7fb272
I20260811 05:13:09.717892 7095 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:13:09.719643 7095 catalog_manager.cc:1347] Generated new cluster ID: daf68a52b610469aa8fdbb69ebcbb84e
I20260811 05:13:09.719702 7095 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:13:09.732275 7095 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:13:09.732963 7095 catalog_manager.cc:1504] Loading token signing keys...
Aug 11 05:13:09 dist-test-slave-f7th krb5kdc[6910](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425189, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:13:09.737326 7095 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0580d2ae5a674b82993e5dae8978293c: Generated new TSK 0
I20260811 05:13:09.737519 7095 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:13:09.740796 7110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:09.717960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35506 (local address 127.28.143.190:42997)
0811 05:13:09.718142 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:09.718151 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:09.728910 (+ 10759us) server_negotiation.cc:407] Connection header received
0811 05:13:09.729086 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:09.729094 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:09.729162 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:09.729408 (+ 246us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:09.730174 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:09.731588 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:09.732708 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:09.733328 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:09.733435 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:09.736334 (+ 2899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:09.736340 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:09.736340 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:09.736355 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:09.739074 (+ 2719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:09.739435 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:09.739438 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:09.739439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:09.739570 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:09.739806 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:09.739807 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:09.739808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:09.740103 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:09.740261 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:09.740330 (+ 69us) server_negotiation.cc:299] Negotiation successful
0811 05:13:09.740507 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":93,"threads_started":1}
WARNING: no policy specified for kudu/127.28.143.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab.
Entry for principal kudu/127.28.143.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab.
I20260811 05:13:09.776388 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.129:0
--local_ip_for_outbound_sockets=127.28.143.129
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:42997
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:38267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:09.880903 7119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:09.881165 7119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:09.881192 7119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:09.882543 7119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:09.882618 7119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:09.882650 7119 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.129
I20260811 05:13:09.884737 7119 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:38267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:42997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.129
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:09.885048 7119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:09.885272 7119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:09.891338 7127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:09.891763 7128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:09.891798 7132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:09.892019 7119 server_base.cc:1034] running on GCE node
I20260811 05:13:09.892237 7119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:09.892522 7119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:09.893700 7119 hybrid_clock.cc:648] HybridClock initialized: now 1786425189893650 us; error 74 us; skew 500 ppm
Aug 11 05:13:09 dist-test-slave-f7th krb5kdc[6910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425189, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:09.896508 7119 init.cc:377] Logged in from keytab as kudu/127.28.143.129@KRBTEST.COM (short username kudu)
I20260811 05:13:09.897020 7119 webserver.cc:460] Webserver started at http://127.28.143.129:36593/ using document root <none> and password file <none>
I20260811 05:13:09.897246 7119 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:09.897313 7119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:09.897428 7119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:09.898533 7119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-0/data/instance:
uuid: "cc4d288f98ea45b39b3724d93dd61907"
format_stamp: "Formatted at 2026-08-11 05:13:09 on dist-test-slave-f7th"
server_key: "abfac713b11b4be80e937497ee6c649a"
server_key_iv: "5c9f85fc371d29c2a7036cfe05d2c1c3"
server_key_version: "encryptionkey@0"
I20260811 05:13:09.898923 7119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance:
uuid: "cc4d288f98ea45b39b3724d93dd61907"
format_stamp: "Formatted at 2026-08-11 05:13:09 on dist-test-slave-f7th"
server_key: "abfac713b11b4be80e937497ee6c649a"
server_key_iv: "5c9f85fc371d29c2a7036cfe05d2c1c3"
server_key_version: "encryptionkey@0"
I20260811 05:13:09.900462 7119 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 05:13:09.901445 7145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:09.901654 7119 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:09.901765 7119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-0/wal
uuid: "cc4d288f98ea45b39b3724d93dd61907"
format_stamp: "Formatted at 2026-08-11 05:13:09 on dist-test-slave-f7th"
server_key: "abfac713b11b4be80e937497ee6c649a"
server_key_iv: "5c9f85fc371d29c2a7036cfe05d2c1c3"
server_key_version: "encryptionkey@0"
I20260811 05:13:09.901841 7119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:09.931808 7119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:09.934166 7119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:09.934329 7119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:09.934592 7119 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:09.934927 7119 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:09.934964 7119 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:09.934995 7119 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:09.935012 7119 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:09.940852 7119 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.129:38369
I20260811 05:13:09.940898 7371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.129:38369 every 8 connection(s)
I20260811 05:13:09.941227 7119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
Aug 11 05:13:09 dist-test-slave-f7th krb5kdc[6910](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425189, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:13:09.948032 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 7119
I20260811 05:13:09.948171 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance
I20260811 05:13:09.948344 29246 external_mini_cluster.cc:1428] Setting key 81d0ed399b3161c224b95ebdc4464eb0
I20260811 05:13:09.951372 7110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:09.942000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:37011 (local address 127.28.143.190:42997)
0811 05:13:09.942135 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:09.942137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:09.942305 (+ 168us) server_negotiation.cc:407] Connection header received
0811 05:13:09.943316 (+ 1011us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:09.943317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:09.943337 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:09.943408 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:09.944083 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:09.944588 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:09.945635 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:09.945850 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:09.945908 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:09.948398 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:09.948401 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:09.948401 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:09.948411 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:09.949960 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:09.950390 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:09.950391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:09.950392 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:09.950458 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:09.950698 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:09.950699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:09.950699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:09.950825 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:09.950883 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:09.951168 (+ 285us) server_negotiation.cc:299] Negotiation successful
0811 05:13:09.951277 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260811 05:13:09.951642 7378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:09.941931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:42997 (local address 127.28.143.129:37011)
0811 05:13:09.942170 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:09.942215 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:09.943192 (+ 977us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:09.943497 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:09.943501 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:09.943777 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:09.943982 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:09.943986 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:09.944699 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:09.944700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:09.945535 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:09.945538 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:09.945930 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:09.945931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:09.946058 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:09.946700 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:09.946710 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:09.948279 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:09.950049 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:09.950061 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:09.950064 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:09.950316 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:09.950514 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:09.950515 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:09.950515 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:09.950639 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:09.950881 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:09.950890 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:09.951042 (+ 152us) client_negotiation.cc:769] Sending connection context
0811 05:13:09.951137 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 05:13:09.951310 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":94,"threads_started":1}
I20260811 05:13:09.952265 7372 heartbeater.cc:347] Connected to a master server at 127.28.143.190:42997
I20260811 05:13:09.952382 7372 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:09.952721 7372 heartbeater.cc:511] Master 127.28.143.190:42997 requested a full tablet report, sending...
I20260811 05:13:09.953436 7009 ts_manager.cc:194] Registered new tserver with Master: cc4d288f98ea45b39b3724d93dd61907 (127.28.143.129:38369)
I20260811 05:13:09.954408 7009 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.129@KRBTEST.COM'} at 127.28.143.129:37011
WARNING: no policy specified for kudu/127.28.143.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab.
Entry for principal kudu/127.28.143.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab.
I20260811 05:13:09.980406 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.130:0
--local_ip_for_outbound_sockets=127.28.143.130
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:42997
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:38267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:10.071368 7385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:10.071607 7385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:10.071643 7385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:10.072964 7385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:10.073019 7385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:10.073056 7385 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.130
I20260811 05:13:10.074640 7385 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:38267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:42997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.130
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:10.074940 7385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:10.075189 7385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:10.080272 7393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:10.080272 7397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:10.080291 7394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:10.080689 7385 server_base.cc:1034] running on GCE node
I20260811 05:13:10.080945 7385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:10.081176 7385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:10.082355 7385 hybrid_clock.cc:648] HybridClock initialized: now 1786425190082343 us; error 34 us; skew 500 ppm
Aug 11 05:13:10 dist-test-slave-f7th krb5kdc[6910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425190, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:10.085175 7385 init.cc:377] Logged in from keytab as kudu/127.28.143.130@KRBTEST.COM (short username kudu)
I20260811 05:13:10.085668 7385 webserver.cc:460] Webserver started at http://127.28.143.130:41335/ using document root <none> and password file <none>
I20260811 05:13:10.085919 7385 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:10.085973 7385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:10.086104 7385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:10.087268 7385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-1/data/instance:
uuid: "2de02e96081e43aaae1f256d2d4f6405"
format_stamp: "Formatted at 2026-08-11 05:13:10 on dist-test-slave-f7th"
server_key: "2ee6ca932927190fdec95588f1c2b3df"
server_key_iv: "e3b0d2cc11909aa5941c60201eca59ea"
server_key_version: "encryptionkey@0"
I20260811 05:13:10.087630 7385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance:
uuid: "2de02e96081e43aaae1f256d2d4f6405"
format_stamp: "Formatted at 2026-08-11 05:13:10 on dist-test-slave-f7th"
server_key: "2ee6ca932927190fdec95588f1c2b3df"
server_key_iv: "e3b0d2cc11909aa5941c60201eca59ea"
server_key_version: "encryptionkey@0"
I20260811 05:13:10.089115 7385 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 05:13:10.089959 7406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:10.090118 7385 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 05:13:10.090215 7385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-1/wal
uuid: "2de02e96081e43aaae1f256d2d4f6405"
format_stamp: "Formatted at 2026-08-11 05:13:10 on dist-test-slave-f7th"
server_key: "2ee6ca932927190fdec95588f1c2b3df"
server_key_iv: "e3b0d2cc11909aa5941c60201eca59ea"
server_key_version: "encryptionkey@0"
I20260811 05:13:10.090297 7385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:10.103789 7385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:10.106088 7385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:10.106231 7385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:10.106456 7385 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:10.106788 7385 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:10.106823 7385 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:10.106873 7385 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:10.106897 7385 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:10.112447 7385 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.130:41445
I20260811 05:13:10.112489 7622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.130:41445 every 8 connection(s)
I20260811 05:13:10.112851 7385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
I20260811 05:13:10.116875 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 7385
I20260811 05:13:10.117022 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance
I20260811 05:13:10.117192 29246 external_mini_cluster.cc:1428] Setting key 04cce0b9030d3325f4e37fa2dbe899f5
Aug 11 05:13:10 dist-test-slave-f7th krb5kdc[6910](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425190, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:13:10.123044 7110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:10.113614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:36739 (local address 127.28.143.190:42997)
0811 05:13:10.113696 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:10.113699 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:10.113893 (+ 194us) server_negotiation.cc:407] Connection header received
0811 05:13:10.114916 (+ 1023us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:10.114918 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:10.114931 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:10.115002 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:10.115666 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:10.116236 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:10.117440 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:10.117671 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:10.117725 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:10.120100 (+ 2375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:10.120102 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:10.120102 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:10.120111 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:10.121658 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:10.122097 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:10.122098 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:10.122098 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:10.122174 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:10.122389 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:10.122390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:10.122390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:10.122544 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:10.122615 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:10.122849 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 05:13:10.122942 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 05:13:10.123164 7629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:10.113592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:42997 (local address 127.28.143.130:36739)
0811 05:13:10.113755 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:10.113801 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:10.114765 (+ 964us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:10.115069 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:10.115073 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:10.115390 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:10.115595 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:10.115599 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:10.116338 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:10.116339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:10.117349 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:10.117352 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:10.117763 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:10.117764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:10.117878 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:10.118457 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:10.118467 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:10.119998 (+ 1531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:10.121754 (+ 1756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:10.121760 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:10.121762 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:10.122021 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:10.122240 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:10.122240 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:10.122240 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:10.122333 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:10.122604 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:10.122609 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:10.122731 (+ 122us) client_negotiation.cc:769] Sending connection context
0811 05:13:10.122814 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 05:13:10.122972 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":77,"threads_started":1}
I20260811 05:13:10.123670 7627 heartbeater.cc:347] Connected to a master server at 127.28.143.190:42997
I20260811 05:13:10.123775 7627 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:10.123960 7627 heartbeater.cc:511] Master 127.28.143.190:42997 requested a full tablet report, sending...
I20260811 05:13:10.124379 7009 ts_manager.cc:194] Registered new tserver with Master: 2de02e96081e43aaae1f256d2d4f6405 (127.28.143.130:41445)
I20260811 05:13:10.124914 7009 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.130@KRBTEST.COM'} at 127.28.143.130:36739
WARNING: no policy specified for kudu/127.28.143.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab.
Entry for principal kudu/127.28.143.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab.
I20260811 05:13:10.149140 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.131:0
--local_ip_for_outbound_sockets=127.28.143.131
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:42997
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:38267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:10.239661 7632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:10.239905 7632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:10.239943 7632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:10.241281 7632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:10.241340 7632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:10.241381 7632 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.131
I20260811 05:13:10.243052 7632 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:38267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:42997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.131
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:10.243322 7632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:10.243567 7632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:10.249130 7647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:10.249114 7640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:10.249120 7641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:10.249231 7632 server_base.cc:1034] running on GCE node
I20260811 05:13:10.249606 7632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:10.249872 7632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:10.251061 7632 hybrid_clock.cc:648] HybridClock initialized: now 1786425190251037 us; error 39 us; skew 500 ppm
Aug 11 05:13:10 dist-test-slave-f7th krb5kdc[6910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425190, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:10.254012 7632 init.cc:377] Logged in from keytab as kudu/127.28.143.131@KRBTEST.COM (short username kudu)
I20260811 05:13:10.254546 7632 webserver.cc:460] Webserver started at http://127.28.143.131:45219/ using document root <none> and password file <none>
I20260811 05:13:10.254767 7632 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:10.254818 7632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:10.254956 7632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:10.256078 7632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-2/data/instance:
uuid: "22e8cafa65354dc3882c17172212b1e7"
format_stamp: "Formatted at 2026-08-11 05:13:10 on dist-test-slave-f7th"
server_key: "5c3dc65ac18d2725462e1840a06bb955"
server_key_iv: "2d8a3878b4444ce3af2bbdfa721e5980"
server_key_version: "encryptionkey@0"
I20260811 05:13:10.256422 7632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance:
uuid: "22e8cafa65354dc3882c17172212b1e7"
format_stamp: "Formatted at 2026-08-11 05:13:10 on dist-test-slave-f7th"
server_key: "5c3dc65ac18d2725462e1840a06bb955"
server_key_iv: "2d8a3878b4444ce3af2bbdfa721e5980"
server_key_version: "encryptionkey@0"
I20260811 05:13:10.257934 7632 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 05:13:10.258909 7663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:10.259159 7632 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:10.259264 7632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-2/wal
uuid: "22e8cafa65354dc3882c17172212b1e7"
format_stamp: "Formatted at 2026-08-11 05:13:10 on dist-test-slave-f7th"
server_key: "5c3dc65ac18d2725462e1840a06bb955"
server_key_iv: "2d8a3878b4444ce3af2bbdfa721e5980"
server_key_version: "encryptionkey@0"
I20260811 05:13:10.259346 7632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:10.296797 7632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:10.299134 7632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:10.299285 7632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:10.299539 7632 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:10.299866 7632 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:10.299901 7632 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:10.299932 7632 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:10.299948 7632 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:10.305750 7632 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.131:32941
I20260811 05:13:10.305816 7878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.131:32941 every 8 connection(s)
I20260811 05:13:10.306183 7632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
Aug 11 05:13:10 dist-test-slave-f7th krb5kdc[6910](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425190, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:13:10.316357 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 7632
I20260811 05:13:10.316481 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance
I20260811 05:13:10.316671 29246 external_mini_cluster.cc:1428] Setting key 7617ec70eba70d0f6c04326a8a41937f
I20260811 05:13:10.316790 7110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:10.307006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:54873 (local address 127.28.143.190:42997)
0811 05:13:10.307100 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:10.307102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:10.307247 (+ 145us) server_negotiation.cc:407] Connection header received
0811 05:13:10.308226 (+ 979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:10.308227 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:10.308240 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:10.308296 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:10.308996 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:10.309542 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:10.310584 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:10.310793 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:10.310864 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:10.313465 (+ 2601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:10.313467 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:10.313467 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:10.313477 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:10.314962 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:10.315456 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:10.315458 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:10.315458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:10.315524 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:10.315773 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:10.315774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:10.315775 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:10.316112 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:10.316195 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:10.316552 (+ 357us) server_negotiation.cc:299] Negotiation successful
0811 05:13:10.316685 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 05:13:10.317368 7885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:10.306928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:42997 (local address 127.28.143.131:54873)
0811 05:13:10.307110 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:10.307160 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:10.308113 (+ 953us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:10.308355 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:10.308358 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:10.308690 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:10.308898 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:10.308903 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:10.309643 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:10.309644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:10.310476 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:10.310479 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:10.310905 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:10.310906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:10.311027 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:10.311577 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:10.311586 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:10.313324 (+ 1738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:10.315056 (+ 1732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:10.315066 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:10.315069 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:10.315353 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:10.315583 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:10.315583 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:10.315584 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:10.315711 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:10.316198 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:10.316208 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:10.316374 (+ 166us) client_negotiation.cc:769] Sending connection context
0811 05:13:10.316891 (+ 517us) client_negotiation.cc:240] Negotiation successful
0811 05:13:10.317152 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":75,"threads_started":1}
I20260811 05:13:10.317813 7880 heartbeater.cc:347] Connected to a master server at 127.28.143.190:42997
I20260811 05:13:10.317920 7880 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:10.318084 7880 heartbeater.cc:511] Master 127.28.143.190:42997 requested a full tablet report, sending...
I20260811 05:13:10.318476 7009 ts_manager.cc:194] Registered new tserver with Master: 22e8cafa65354dc3882c17172212b1e7 (127.28.143.131:32941)
I20260811 05:13:10.318873 7009 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.131@KRBTEST.COM'} at 127.28.143.131:54873
I20260811 05:13:10.328787 29246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/krb5kdc/test-admin.keytab.
Aug 11 05:13:10 dist-test-slave-f7th krb5kdc[6910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:10.349251 29246 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 05:13:10.351105 7898 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:10.350212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57732 (local address 127.28.143.129:38369)
0811 05:13:10.350377 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:10.350382 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:10.350390 (+ 8us) server_negotiation.cc:407] Connection header received
0811 05:13:10.350417 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:10.350419 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:10.350433 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:10.350498 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:10.351019 (+ 521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57732: BlockingRecv error: recv got EOF from 127.0.0.1:57732 (error 108)
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":79,"threads_started":1}
W20260811 05:13:10.351279 29246 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/krb5kdc/krb5cc' of type FILE
Aug 11 05:13:10 dist-test-slave-f7th krb5kdc[6910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:10.352413 29246 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 05:13:10 dist-test-slave-f7th krb5kdc[6910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:10.353533 29246 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 05:13:10 dist-test-slave-f7th krb5kdc[6910](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.143.129@KRBTEST.COM
I20260811 05:13:10.362154 7898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:10.354199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57740 (local address 127.28.143.129:38369)
0811 05:13:10.354254 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:10.354256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:10.354270 (+ 14us) server_negotiation.cc:407] Connection header received
0811 05:13:10.354360 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:10.354361 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:10.354371 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:10.354420 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:10.354851 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:10.355610 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:10.356333 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:10.356713 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:10.356772 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:10.359043 (+ 2271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:10.359054 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:10.359055 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:10.359063 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:10.360962 (+ 1899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:10.361331 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:10.361334 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:10.361335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:10.361394 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:10.361569 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:10.361570 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:10.361570 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:10.361728 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:10.361906 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:10.361953 (+ 47us) server_negotiation.cc:299] Negotiation successful
0811 05:13:10.362045 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
W20260811 05:13:10.364300 7898 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:10.363711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57750 (local address 127.28.143.129:38369)
0811 05:13:10.363763 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:10.363765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:10.363786 (+ 21us) server_negotiation.cc:407] Connection header received
0811 05:13:10.363859 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:10.363859 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:10.363871 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:10.363911 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:10.364248 (+ 337us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57750: BlockingRecv error: recv got EOF from 127.0.0.1:57750 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260811 05:13:10.364336 29246 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/krb5kdc/krb5cc' of type FILE
Aug 11 05:13:10 dist-test-slave-f7th krb5kdc[6910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:10.365427 29246 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 05:13:10 dist-test-slave-f7th krb5kdc[6910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:10.366394 29246 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 05:13:10 dist-test-slave-f7th krb5kdc[6910](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.143.129@KRBTEST.COM
I20260811 05:13:10.374847 7898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:10.367067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57762 (local address 127.28.143.129:38369)
0811 05:13:10.367157 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:10.367159 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:10.367228 (+ 69us) server_negotiation.cc:407] Connection header received
0811 05:13:10.367296 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:10.367297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:10.367308 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:10.367354 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:10.367881 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:10.368484 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:10.369268 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:10.369478 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:10.369537 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:10.371945 (+ 2408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:10.371947 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:10.371947 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:10.371956 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:10.373435 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:10.373825 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:10.373826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:10.373826 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:10.373886 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:10.374215 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:10.374216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:10.374217 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:10.374340 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:10.374432 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:10.374643 (+ 211us) server_negotiation.cc:299] Negotiation successful
0811 05:13:10.374741 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260811 05:13:10.376658 7898 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:10.375986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57776 (local address 127.28.143.129:38369)
0811 05:13:10.376063 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:10.376064 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:10.376119 (+ 55us) server_negotiation.cc:407] Connection header received
0811 05:13:10.376166 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:10.376167 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:10.376175 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:10.376216 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:10.376577 (+ 361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57776: BlockingRecv error: recv got EOF from 127.0.0.1:57776 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260811 05:13:10.376739 29246 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425159060450-29246-0/krb5kdc/krb5cc' of type FILE
Aug 11 05:13:10 dist-test-slave-f7th krb5kdc[6910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:10.377816 29246 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 05:13:10 dist-test-slave-f7th krb5kdc[6910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:10.378990 29246 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 05:13:10 dist-test-slave-f7th krb5kdc[6910](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.143.129@KRBTEST.COM
I20260811 05:13:10.386502 7898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:10.379660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57788 (local address 127.28.143.129:38369)
0811 05:13:10.379764 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:10.379766 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:10.379776 (+ 10us) server_negotiation.cc:407] Connection header received
0811 05:13:10.379822 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:10.379823 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:10.379835 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:10.379878 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:10.380329 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:10.380870 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:10.381587 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:10.381779 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:10.381838 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:10.384151 (+ 2313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:10.384153 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:10.384153 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:10.384161 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:10.385379 (+ 1218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:10.385705 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:10.385706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:10.385707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:10.385762 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:10.385950 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:10.385951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:10.385951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:10.386065 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:10.386159 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:10.386282 (+ 123us) server_negotiation.cc:299] Negotiation successful
0811 05:13:10.386416 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 05:13:10.387660 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 7119
I20260811 05:13:10.402550 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 7385
I20260811 05:13:10.419974 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 7632
I20260811 05:13:10.442481 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 6936
2026-08-11T05:13:10Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2252 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:13:10 dist-test-slave-f7th krb5kdc[7922](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:13:10 dist-test-slave-f7th krb5kdc[7922](info): set up 2 sockets
Aug 11 05:13:10 dist-test-slave-f7th krb5kdc[7922](info): commencing operation
krb5kdc: starting...
W20260811 05:13:11.689955 29246 mini_kdc.cc:121] Time spent starting KDC: real 1.209s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Aug 11 05:13:11 dist-test-slave-f7th krb5kdc[7922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425191, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:13:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:13:11Z Disabled control of system clock
WARNING: no policy specified for oryx/127.28.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.190.keytab.
Entry for principal oryx/127.28.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.190.keytab.
I20260811 05:13:11.832736 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:39435
--webserver_interface=127.28.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.190.keytab
--principal=oryx/127.28.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:43653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:39435
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:11.926960 7946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:11.927207 7946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:11.927232 7946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:11.929371 7946 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:11.929445 7946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:11.929461 7946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:11.929473 7946 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:11.929486 7946 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:13:11.931493 7946 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:43653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:39435
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.190.keytab
--principal=oryx/127.28.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:39435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:11.931818 7946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:11.932081 7946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:11.937953 7955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:11.937964 7952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:11.938050 7946 server_base.cc:1034] running on GCE node
W20260811 05:13:11.938007 7959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:11.938462 7946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:11.938781 7946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:11.939989 7946 hybrid_clock.cc:648] HybridClock initialized: now 1786425191939972 us; error 34 us; skew 500 ppm
Aug 11 05:13:11 dist-test-slave-f7th krb5kdc[7922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425191, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:11.943132 7946 init.cc:377] Logged in from keytab as oryx/127.28.143.190@KRBTEST.COM (short username oryx)
I20260811 05:13:11.943662 7946 webserver.cc:460] Webserver started at http://127.28.143.190:42883/ using document root <none> and password file <none>
I20260811 05:13:11.943889 7946 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:11.943938 7946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:11.944068 7946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:11.945297 7946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "dc6e53c73bdd4fe18d564e13636cdd5b"
format_stamp: "Formatted at 2026-08-11 05:13:11 on dist-test-slave-f7th"
server_key: "0e08f880a17cd1ade0ed7c8e8b9bd44f"
server_key_iv: "58fd9832618f1fa2f73be12810289eaf"
server_key_version: "encryptionkey@0"
I20260811 05:13:11.945669 7946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "dc6e53c73bdd4fe18d564e13636cdd5b"
format_stamp: "Formatted at 2026-08-11 05:13:11 on dist-test-slave-f7th"
server_key: "0e08f880a17cd1ade0ed7c8e8b9bd44f"
server_key_iv: "58fd9832618f1fa2f73be12810289eaf"
server_key_version: "encryptionkey@0"
I20260811 05:13:11.947548 7946 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260811 05:13:11.948534 7976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:11.948773 7946 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:11.948882 7946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "dc6e53c73bdd4fe18d564e13636cdd5b"
format_stamp: "Formatted at 2026-08-11 05:13:11 on dist-test-slave-f7th"
server_key: "0e08f880a17cd1ade0ed7c8e8b9bd44f"
server_key_iv: "58fd9832618f1fa2f73be12810289eaf"
server_key_version: "encryptionkey@0"
I20260811 05:13:11.948956 7946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:11.967165 7946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:11.970465 7946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:11.970654 7946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:11.975385 7946 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:39435
I20260811 05:13:11.975445 8086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:39435 every 8 connection(s)
I20260811 05:13:11.975857 7946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:13:11.977087 8087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:11.979575 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 7946
I20260811 05:13:11.979720 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:13:11.979914 29246 external_mini_cluster.cc:1428] Setting key 2422d2aa8b56fb87cac756a4a1b1fe65
I20260811 05:13:11.980193 8087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b: Bootstrap starting.
I20260811 05:13:11.980993 8087 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:11.981287 8087 log.cc:824] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:11.982084 8087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b: No bootstrap required, opened a new log
I20260811 05:13:11.984455 8087 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } }
I20260811 05:13:11.984712 8087 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:11.984750 8087 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc6e53c73bdd4fe18d564e13636cdd5b, State: Initialized, Role: FOLLOWER
I20260811 05:13:11.984934 8087 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } }
I20260811 05:13:11.985065 8087 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:13:11.985111 8087 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:13:11.985160 8087 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:11.986013 8087 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } }
I20260811 05:13:11.986122 8087 leader_election.cc:302] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc6e53c73bdd4fe18d564e13636cdd5b; no voters:
I20260811 05:13:11.986646 8087 leader_election.cc:288] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:13:11.986791 8100 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:11.987159 8087 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:13:11.987459 8100 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [term 1 LEADER]: Becoming Leader. State: Replica: dc6e53c73bdd4fe18d564e13636cdd5b, State: Running, Role: LEADER
I20260811 05:13:11.987653 8100 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } }
I20260811 05:13:11.988181 8104 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc6e53c73bdd4fe18d564e13636cdd5b. Latest consensus state: current_term: 1 leader_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } }
I20260811 05:13:11.988272 8104 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:11.988315 8101 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } }
I20260811 05:13:11.988433 8101 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:11.988678 8115 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 11 05:13:11 dist-test-slave-f7th krb5kdc[7922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425191, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM
I20260811 05:13:11.989661 8115 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:13:11.992822 8115 catalog_manager.cc:1347] Generated new cluster ID: bbcde4e7dda24824a0a9e147d6d1f4b1
I20260811 05:13:11.992957 8115 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:13:11.994457 8097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:11.980892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38100 (local address 127.28.143.190:39435)
0811 05:13:11.981118 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:11.981129 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:11.981165 (+ 36us) server_negotiation.cc:407] Connection header received
0811 05:13:11.981745 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:11.981753 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:11.981812 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:11.981958 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:11.982825 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:11.983813 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:11.984988 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:11.985622 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:11.985747 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:11.990050 (+ 4303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:11.990056 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:11.990056 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:11.990071 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:11.992154 (+ 2083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:11.992937 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:11.992941 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:11.992942 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:11.993179 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:11.993442 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:11.993442 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:11.993443 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:11.993791 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:11.993958 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:11.994058 (+ 100us) server_negotiation.cc:299] Negotiation successful
0811 05:13:11.994223 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":105,"threads_started":1}
I20260811 05:13:11.998699 8115 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:13:11.999230 8115 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:13:12.002172 8115 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b: Generated new TSK 0
I20260811 05:13:12.002373 8115 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.28.143.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.143.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.143.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.129.keytab.
Entry for principal oryx/127.28.143.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.129.keytab.
I20260811 05:13:12.044035 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.129:0
--local_ip_for_outbound_sockets=127.28.143.129
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:39435
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.129.keytab
--principal=oryx/127.28.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:43653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:12.132874 8136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:12.133236 8136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:12.133286 8136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:12.134727 8136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:12.134779 8136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:12.134809 8136 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.129
I20260811 05:13:12.136415 8136 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:43653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.129.keytab
--principal=oryx/127.28.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:39435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.129
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:12.136701 8136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:12.136950 8136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 05:13:12.142259 8136 server_base.cc:1034] running on GCE node
W20260811 05:13:12.142215 8147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:12.142228 8146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:12.142210 8154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:12.142844 8136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:12.143131 8136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:12.144330 8136 hybrid_clock.cc:648] HybridClock initialized: now 1786425192144305 us; error 42 us; skew 500 ppm
Aug 11 05:13:12 dist-test-slave-f7th krb5kdc[7922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425192, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:12.148154 8136 init.cc:377] Logged in from keytab as oryx/127.28.143.129@KRBTEST.COM (short username oryx)
I20260811 05:13:12.148743 8136 webserver.cc:460] Webserver started at http://127.28.143.129:42899/ using document root <none> and password file <none>
I20260811 05:13:12.148999 8136 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:12.149055 8136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:12.149190 8136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:12.150324 8136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-0/data/instance:
uuid: "4c4cc9aaf7b14d99920b5b6edca2a68a"
format_stamp: "Formatted at 2026-08-11 05:13:12 on dist-test-slave-f7th"
server_key: "4cd0fd5c1783b66a8af65c0baa84897d"
server_key_iv: "d7c3e8930a29ba11b1f0f7a0ae4b18ac"
server_key_version: "encryptionkey@0"
I20260811 05:13:12.150869 8136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance:
uuid: "4c4cc9aaf7b14d99920b5b6edca2a68a"
format_stamp: "Formatted at 2026-08-11 05:13:12 on dist-test-slave-f7th"
server_key: "4cd0fd5c1783b66a8af65c0baa84897d"
server_key_iv: "d7c3e8930a29ba11b1f0f7a0ae4b18ac"
server_key_version: "encryptionkey@0"
I20260811 05:13:12.152335 8136 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 05:13:12.153400 8162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:12.153729 8136 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:12.153846 8136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-0/wal
uuid: "4c4cc9aaf7b14d99920b5b6edca2a68a"
format_stamp: "Formatted at 2026-08-11 05:13:12 on dist-test-slave-f7th"
server_key: "4cd0fd5c1783b66a8af65c0baa84897d"
server_key_iv: "d7c3e8930a29ba11b1f0f7a0ae4b18ac"
server_key_version: "encryptionkey@0"
I20260811 05:13:12.153975 8136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:12.176539 8136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:12.179370 8136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:12.179517 8136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:12.179850 8136 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:12.180262 8136 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:12.180310 8136 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:12.180341 8136 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:12.180352 8136 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:12.187472 8136 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.129:39087
I20260811 05:13:12.187592 8405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.129:39087 every 8 connection(s)
I20260811 05:13:12.187918 8136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
I20260811 05:13:12.190619 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 8136
I20260811 05:13:12.190737 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance
I20260811 05:13:12.190934 29246 external_mini_cluster.cc:1428] Setting key 66fad7763da99c40a0dc762180aea357
Aug 11 05:13:12 dist-test-slave-f7th krb5kdc[7922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425192, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.143.129@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM
I20260811 05:13:12.199618 8097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:12.188895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:50473 (local address 127.28.143.190:39435)
0811 05:13:12.188992 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:12.188994 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:12.189285 (+ 291us) server_negotiation.cc:407] Connection header received
0811 05:13:12.189924 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:12.189925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:12.189940 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:12.189991 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:12.191297 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.191921 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.193437 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.193657 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.193730 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:12.196293 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:12.196295 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:12.196296 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:12.196306 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:12.198129 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:12.198600 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:12.198601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:12.198601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:12.198676 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:12.198907 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:12.198907 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:12.198908 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:12.199074 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:12.199144 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:12.199405 (+ 261us) server_negotiation.cc:299] Negotiation successful
0811 05:13:12.199510 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 05:13:12.199704 8409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:12.188935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.28.143.129:50473)
0811 05:13:12.189152 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:12.189194 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:12.189809 (+ 615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:12.190083 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:12.190087 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:12.190859 (+ 772us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:12.191183 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:12.191190 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.192016 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.192017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:12.193293 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:12.193297 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.193758 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.193759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:12.193914 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:12.194527 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:12.194537 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:12.196171 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:12.198240 (+ 2069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:12.198249 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:12.198251 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:12.198513 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:12.198744 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:12.198745 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:12.198745 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:12.198853 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:12.199144 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:12.199149 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:12.199280 (+ 131us) client_negotiation.cc:769] Sending connection context
0811 05:13:12.199363 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 05:13:12.199505 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":153,"threads_started":1}
I20260811 05:13:12.200282 8406 heartbeater.cc:347] Connected to a master server at 127.28.143.190:39435
I20260811 05:13:12.200412 8406 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:12.200667 8406 heartbeater.cc:511] Master 127.28.143.190:39435 requested a full tablet report, sending...
I20260811 05:13:12.201351 8010 ts_manager.cc:194] Registered new tserver with Master: 4c4cc9aaf7b14d99920b5b6edca2a68a (127.28.143.129:39087)
WARNING: no policy specified for oryx/127.28.143.130@KRBTEST.COM; defaulting to no policy
I20260811 05:13:12.202054 8010 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.143.129@KRBTEST.COM'} at 127.28.143.129:50473
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.143.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.143.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.130.keytab.
Entry for principal oryx/127.28.143.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.130.keytab.
I20260811 05:13:12.224941 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.130:0
--local_ip_for_outbound_sockets=127.28.143.130
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:39435
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.130.keytab
--principal=oryx/127.28.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:43653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:12.311411 8414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:12.311643 8414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:12.311671 8414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:12.312935 8414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:12.312986 8414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:12.313015 8414 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.130
I20260811 05:13:12.314613 8414 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:43653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.130.keytab
--principal=oryx/127.28.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:39435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8414
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.130
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:12.314893 8414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:12.315150 8414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:12.320739 8422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:12.320714 8421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:12.320971 8414 server_base.cc:1034] running on GCE node
W20260811 05:13:12.320714 8426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:12.321265 8414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:12.321496 8414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:12.322665 8414 hybrid_clock.cc:648] HybridClock initialized: now 1786425192322643 us; error 34 us; skew 500 ppm
Aug 11 05:13:12 dist-test-slave-f7th krb5kdc[7922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425192, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:12.325459 8414 init.cc:377] Logged in from keytab as oryx/127.28.143.130@KRBTEST.COM (short username oryx)
I20260811 05:13:12.325994 8414 webserver.cc:460] Webserver started at http://127.28.143.130:41741/ using document root <none> and password file <none>
I20260811 05:13:12.326242 8414 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:12.326293 8414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:12.326426 8414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:12.327544 8414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-1/data/instance:
uuid: "fda127cd3b28435498b97c4445c40438"
format_stamp: "Formatted at 2026-08-11 05:13:12 on dist-test-slave-f7th"
server_key: "cc8c2ee870d4644bd758134621804df8"
server_key_iv: "d65763378af5c9ca329b6536c113f218"
server_key_version: "encryptionkey@0"
I20260811 05:13:12.327888 8414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance:
uuid: "fda127cd3b28435498b97c4445c40438"
format_stamp: "Formatted at 2026-08-11 05:13:12 on dist-test-slave-f7th"
server_key: "cc8c2ee870d4644bd758134621804df8"
server_key_iv: "d65763378af5c9ca329b6536c113f218"
server_key_version: "encryptionkey@0"
I20260811 05:13:12.329357 8414 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 05:13:12.330212 8438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:12.330380 8414 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:12.330482 8414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-1/wal
uuid: "fda127cd3b28435498b97c4445c40438"
format_stamp: "Formatted at 2026-08-11 05:13:12 on dist-test-slave-f7th"
server_key: "cc8c2ee870d4644bd758134621804df8"
server_key_iv: "d65763378af5c9ca329b6536c113f218"
server_key_version: "encryptionkey@0"
I20260811 05:13:12.330549 8414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:12.338982 8414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:12.341192 8414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:12.341336 8414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:12.341569 8414 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:12.341879 8414 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:12.341913 8414 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:12.341943 8414 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:12.341969 8414 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:12.348337 8414 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.130:34837
I20260811 05:13:12.348387 8666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.130:34837 every 8 connection(s)
I20260811 05:13:12.348788 8414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
I20260811 05:13:12.351414 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 8414
I20260811 05:13:12.351524 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance
I20260811 05:13:12.351799 29246 external_mini_cluster.cc:1428] Setting key e6a604c25afe4e61fd72396c0baa67d2
Aug 11 05:13:12 dist-test-slave-f7th krb5kdc[7922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425192, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.143.130@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM
I20260811 05:13:12.361555 8097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:12.349620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:35521 (local address 127.28.143.190:39435)
0811 05:13:12.349668 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:12.349670 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:12.349889 (+ 219us) server_negotiation.cc:407] Connection header received
0811 05:13:12.350416 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:12.350417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:12.350429 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:12.350480 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:12.351459 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.352443 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.353721 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.354041 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.354171 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:12.357230 (+ 3059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:12.357232 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:12.357232 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:12.357241 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:12.359556 (+ 2315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:12.360299 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:12.360301 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:12.360302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:12.360380 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:12.360778 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:12.360780 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:12.360780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:12.360919 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:12.360977 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:12.361388 (+ 411us) server_negotiation.cc:299] Negotiation successful
0811 05:13:12.361482 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 05:13:12.361806 8673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:12.349615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.28.143.130:35521)
0811 05:13:12.349767 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:12.349805 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:12.350323 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:12.350546 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:12.350549 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:12.351141 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:12.351346 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:12.351351 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.352555 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.352556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:12.353613 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:12.353617 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.354223 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.354224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:12.354354 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:12.354955 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:12.354968 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:12.357083 (+ 2115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:12.359711 (+ 2628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:12.359728 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:12.359732 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:12.360140 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:12.360470 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:12.360471 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:12.360472 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:12.360686 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:12.360987 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:12.360995 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:12.361196 (+ 201us) client_negotiation.cc:769] Sending connection context
0811 05:13:12.361319 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 05:13:12.361550 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":85,"threads_started":1}
I20260811 05:13:12.362376 8670 heartbeater.cc:347] Connected to a master server at 127.28.143.190:39435
I20260811 05:13:12.362505 8670 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for oryx/127.28.143.131@KRBTEST.COM; defaulting to no policy
I20260811 05:13:12.362723 8670 heartbeater.cc:511] Master 127.28.143.190:39435 requested a full tablet report, sending...
I20260811 05:13:12.363193 8010 ts_manager.cc:194] Registered new tserver with Master: fda127cd3b28435498b97c4445c40438 (127.28.143.130:34837)
I20260811 05:13:12.363610 8010 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.143.130@KRBTEST.COM'} at 127.28.143.130:35521
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.143.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.143.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.131.keytab.
Entry for principal oryx/127.28.143.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.131.keytab.
I20260811 05:13:12.384876 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.131:0
--local_ip_for_outbound_sockets=127.28.143.131
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:39435
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.131.keytab
--principal=oryx/127.28.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:43653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:12.475459 8680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:12.475704 8680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:12.475729 8680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:12.476920 8680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:12.476960 8680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:12.476995 8680 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.131
I20260811 05:13:12.478513 8680 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:43653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.131.keytab
--principal=oryx/127.28.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:39435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8680
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.131
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:12.478745 8680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:12.478941 8680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:12.484308 8686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:12.484311 8690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:12.484504 8680 server_base.cc:1034] running on GCE node
W20260811 05:13:12.484309 8693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:12.484789 8680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:12.485070 8680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:12.486246 8680 hybrid_clock.cc:648] HybridClock initialized: now 1786425192486223 us; error 37 us; skew 500 ppm
Aug 11 05:13:12 dist-test-slave-f7th krb5kdc[7922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425192, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:12.489411 8680 init.cc:377] Logged in from keytab as oryx/127.28.143.131@KRBTEST.COM (short username oryx)
I20260811 05:13:12.489934 8680 webserver.cc:460] Webserver started at http://127.28.143.131:37113/ using document root <none> and password file <none>
I20260811 05:13:12.490188 8680 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:12.490252 8680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:12.490394 8680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:12.491482 8680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-2/data/instance:
uuid: "b94dae50dc884af4b0d3c50374d3872d"
format_stamp: "Formatted at 2026-08-11 05:13:12 on dist-test-slave-f7th"
server_key: "2f47d269b9480b9e2879b41f53ca01ca"
server_key_iv: "6b9a357e44eefc8ce1e114a9f2309ae7"
server_key_version: "encryptionkey@0"
I20260811 05:13:12.491833 8680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance:
uuid: "b94dae50dc884af4b0d3c50374d3872d"
format_stamp: "Formatted at 2026-08-11 05:13:12 on dist-test-slave-f7th"
server_key: "2f47d269b9480b9e2879b41f53ca01ca"
server_key_iv: "6b9a357e44eefc8ce1e114a9f2309ae7"
server_key_version: "encryptionkey@0"
I20260811 05:13:12.493216 8680 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 05:13:12.493983 8703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:12.494167 8680 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:12.494256 8680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-2/wal
uuid: "b94dae50dc884af4b0d3c50374d3872d"
format_stamp: "Formatted at 2026-08-11 05:13:12 on dist-test-slave-f7th"
server_key: "2f47d269b9480b9e2879b41f53ca01ca"
server_key_iv: "6b9a357e44eefc8ce1e114a9f2309ae7"
server_key_version: "encryptionkey@0"
I20260811 05:13:12.494316 8680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:12.501662 8680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:12.504109 8680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:12.504238 8680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:12.504480 8680 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:12.504837 8680 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:12.504877 8680 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:12.504908 8680 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:12.504932 8680 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:12.510645 8680 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.131:41421
I20260811 05:13:12.510720 8929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.131:41421 every 8 connection(s)
I20260811 05:13:12.511025 8680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
I20260811 05:13:12.511595 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 8680
I20260811 05:13:12.511677 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance
I20260811 05:13:12.511904 29246 external_mini_cluster.cc:1428] Setting key 056df843936221b402539e3579e02be0
Aug 11 05:13:12 dist-test-slave-f7th krb5kdc[7922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425192, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.143.131@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM
I20260811 05:13:12.520812 8097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:12.511877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:57875 (local address 127.28.143.190:39435)
0811 05:13:12.511947 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:12.511949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:12.512226 (+ 277us) server_negotiation.cc:407] Connection header received
0811 05:13:12.512883 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:12.512884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:12.512896 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:12.512937 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:12.513886 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.514398 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.515376 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.515566 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.515621 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:12.518099 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:12.518101 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:12.518101 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:12.518110 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:12.519411 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:12.519812 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:12.519813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:12.519814 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:12.519872 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:12.520116 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:12.520117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:12.520118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:12.520243 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:12.520301 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:12.520598 (+ 297us) server_negotiation.cc:299] Negotiation successful
0811 05:13:12.520715 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 05:13:12.521016 8932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:12.511845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.28.143.131:57875)
0811 05:13:12.512093 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:12.512135 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:12.512792 (+ 657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:12.513005 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:12.513009 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:12.513590 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:12.513791 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:12.513796 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.514482 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.514483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:12.515287 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:12.515290 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.515641 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.515642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:12.515785 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:12.516302 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:12.516312 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:12.517986 (+ 1674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:12.519494 (+ 1508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:12.519502 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:12.519505 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:12.519737 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:12.519934 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:12.519935 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:12.519935 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:12.520056 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:12.520301 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:12.520308 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:12.520469 (+ 161us) client_negotiation.cc:769] Sending connection context
0811 05:13:12.520564 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 05:13:12.520764 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":85,"threads_started":1}
I20260811 05:13:12.521617 8930 heartbeater.cc:347] Connected to a master server at 127.28.143.190:39435
I20260811 05:13:12.521723 8930 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:12.521885 8930 heartbeater.cc:511] Master 127.28.143.190:39435 requested a full tablet report, sending...
I20260811 05:13:12.522277 8010 ts_manager.cc:194] Registered new tserver with Master: b94dae50dc884af4b0d3c50374d3872d (127.28.143.131:41421)
I20260811 05:13:12.522658 8010 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.143.131@KRBTEST.COM'} at 127.28.143.131:57875
I20260811 05:13:12.523679 29246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.28.143.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.143.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.143.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.189.keytab.
Entry for principal oryx/127.28.143.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.189.keytab.
I20260811 05:13:12.554077 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.189:40933
--webserver_interface=127.28.143.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.189.keytab
--principal=oryx/127.28.143.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:43653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:39435,127.28.143.189:40933
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:12.643170 8937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:12.643402 8937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:12.643425 8937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:12.644649 8937 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:12.644694 8937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:12.644709 8937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:12.644721 8937 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:12.644732 8937 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:13:12.646298 8937 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:43653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:39435,127.28.143.189:40933
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.189.keytab
--principal=oryx/127.28.143.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.189:40933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.28.143.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8937
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
W20260811 05:13:12.646438 8937 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.28.143.190:39435,127.28.143.189:40933'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 05:13:12.646565 8937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:12.646761 8937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:12.652447 8942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:12.652577 8946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:12.652447 8950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:12.652689 8937 server_base.cc:1034] running on GCE node
I20260811 05:13:12.653072 8937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:12.653403 8937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:12.654629 8937 hybrid_clock.cc:648] HybridClock initialized: now 1786425192654578 us; error 62 us; skew 500 ppm
Aug 11 05:13:12 dist-test-slave-f7th krb5kdc[7922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425192, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.143.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:12.657485 8937 init.cc:377] Logged in from keytab as oryx/127.28.143.189@KRBTEST.COM (short username oryx)
I20260811 05:13:12.658044 8937 webserver.cc:460] Webserver started at http://127.28.143.189:43859/ using document root <none> and password file <none>
I20260811 05:13:12.658267 8937 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:12.658314 8937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:12.658433 8937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:12.659601 8937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-1/data/instance:
uuid: "db647a2f54b34f5f8f2bd352ce63e7d9"
format_stamp: "Formatted at 2026-08-11 05:13:12 on dist-test-slave-f7th"
server_key: "28c94de5a92ec13f00acceb7d4799029"
server_key_iv: "6dc3cfdb2d98b2ec3cde12c003f1e126"
server_key_version: "encryptionkey@0"
I20260811 05:13:12.659951 8937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-1/wal/instance:
uuid: "db647a2f54b34f5f8f2bd352ce63e7d9"
format_stamp: "Formatted at 2026-08-11 05:13:12 on dist-test-slave-f7th"
server_key: "28c94de5a92ec13f00acceb7d4799029"
server_key_iv: "6dc3cfdb2d98b2ec3cde12c003f1e126"
server_key_version: "encryptionkey@0"
I20260811 05:13:12.661469 8937 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 05:13:12.662326 8962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:12.662521 8937 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:12.662621 8937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-1/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-1/wal
uuid: "db647a2f54b34f5f8f2bd352ce63e7d9"
format_stamp: "Formatted at 2026-08-11 05:13:12 on dist-test-slave-f7th"
server_key: "28c94de5a92ec13f00acceb7d4799029"
server_key_iv: "6dc3cfdb2d98b2ec3cde12c003f1e126"
server_key_version: "encryptionkey@0"
I20260811 05:13:12.662683 8937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:12.675864 8937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:12.677978 8937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:12.678128 8937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:12.682106 8937 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.189:40933
I20260811 05:13:12.682149 9080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.189:40933 every 8 connection(s)
I20260811 05:13:12.682503 8937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-1/data/info.pb
I20260811 05:13:12.683655 9081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:12.685325 9081 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:13:12.691136 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 8937
Aug 11 05:13:12 dist-test-slave-f7th krb5kdc[7922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425192, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.143.189@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM
I20260811 05:13:12.694902 8097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:12.686086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38106 (local address 127.28.143.190:39435)
0811 05:13:12.686135 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:12.686137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:12.686338 (+ 201us) server_negotiation.cc:407] Connection header received
0811 05:13:12.686781 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:12.686782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:12.686792 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:12.686863 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:12.687489 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.688053 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.689289 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.689483 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.689541 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:12.692027 (+ 2486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:12.692029 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:12.692029 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:12.692039 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:12.693531 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:12.693928 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:12.693929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:12.693929 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:12.693986 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:12.694211 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:12.694212 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:12.694212 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:12.694370 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:12.694429 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:12.694702 (+ 273us) server_negotiation.cc:299] Negotiation successful
0811 05:13:12.694793 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 05:13:12.695067 9083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:12.686019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.0.0.1:38106)
0811 05:13:12.686221 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:12.686265 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:12.686703 (+ 438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:12.686917 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:12.686921 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:12.687199 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:12.687392 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:12.687396 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.688169 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.688171 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:12.689192 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:12.689195 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.689557 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.689559 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:12.689684 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:12.690299 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:12.690309 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:12.691914 (+ 1605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:12.693617 (+ 1703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:12.693625 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:12.693627 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:12.693858 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:12.694044 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:12.694045 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:12.694045 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:12.694155 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:12.694433 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:12.694439 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:12.694580 (+ 141us) client_negotiation.cc:769] Sending connection context
0811 05:13:12.694672 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 05:13:12.694829 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":109,"threads_started":1}
I20260811 05:13:12.695848 9081 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.143.189" port: 40933 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 05:13:12 dist-test-slave-f7th krb5kdc[7922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425191, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.28.143.189@KRBTEST.COM
Aug 11 05:13:12 dist-test-slave-f7th krb5kdc[7922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425192, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.143.189@KRBTEST.COM for oryx/127.28.143.189@KRBTEST.COM
I20260811 05:13:12.700781 9086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:12.691679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42416 (local address 127.28.143.189:40933)
0811 05:13:12.691869 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:12.691874 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:12.691901 (+ 27us) server_negotiation.cc:407] Connection header received
0811 05:13:12.691960 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:12.691964 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:12.691978 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:12.692044 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:12.692743 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.693447 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.694166 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.694492 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.694552 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:12.697546 (+ 2994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:12.697554 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:12.697554 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:12.697561 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:12.699292 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:12.699711 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:12.699714 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:12.699715 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:12.699788 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:12.700070 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:12.700071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:12.700072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:12.700278 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:12.700475 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:12.700569 (+ 94us) server_negotiation.cc:299] Negotiation successful
0811 05:13:12.700669 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":87,"threads_started":1}
I20260811 05:13:12.703207 9083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:12.696101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.189:40933 (local address 127.0.0.1:42422)
0811 05:13:12.696147 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:12.696158 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:12.696195 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:12.696462 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:12.696463 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:12.696578 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:12.696657 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:12.696659 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.697549 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.697550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:12.698159 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:12.698162 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.698638 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.698640 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:12.698714 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:12.699047 (+ 333us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:12.699053 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:12.700601 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:12.702348 (+ 1747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:12.702349 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:12.702350 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:12.702519 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:12.702735 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:12.702736 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:12.702736 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:12.702795 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:12.702980 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:12.702981 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:12.703016 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 05:13:12.703065 (+ 49us) client_negotiation.cc:240] Negotiation successful
0811 05:13:12.703143 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260811 05:13:12.703308 9087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:12.696134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42422 (local address 127.28.143.189:40933)
0811 05:13:12.696312 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:12.696314 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:12.696322 (+ 8us) server_negotiation.cc:407] Connection header received
0811 05:13:12.696366 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:12.696367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:12.696378 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:12.696451 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:12.696732 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.697472 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.698247 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.698543 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.698614 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:12.700705 (+ 2091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:12.700706 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:12.700706 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:12.700712 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:12.702254 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:12.702609 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:12.702610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:12.702610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:12.702671 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:12.702839 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:12.702839 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:12.702840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:12.702936 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:12.702983 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:12.703096 (+ 113us) server_negotiation.cc:299] Negotiation successful
0811 05:13:12.703194 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":153,"thread_start_us":109,"threads_started":1}
I20260811 05:13:12.704594 9081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9: Bootstrap starting.
I20260811 05:13:12.705248 9081 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:12.705516 9081 log.cc:824] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:12.706244 9081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9: No bootstrap required, opened a new log
I20260811 05:13:12.707674 9081 raft_consensus.cc:348] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 40933 } }
I20260811 05:13:12.707798 9081 raft_consensus.cc:374] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:12.707820 9081 raft_consensus.cc:729] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db647a2f54b34f5f8f2bd352ce63e7d9, State: Initialized, Role: FOLLOWER
I20260811 05:13:12.707933 9081 consensus_queue.cc:260] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 40933 } }
I20260811 05:13:12.708185 9089 sys_catalog.cc:455] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 40933 } } }
I20260811 05:13:12.708238 9081 sys_catalog.cc:564] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:13:12.708287 9089 sys_catalog.cc:458] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 05:13:12.710311 9113 catalog_manager.cc:1558] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:13:12.710373 9113 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:13:12.712222 8937 master_runner.cc:428] Detected that this master db647a2f54b34f5f8f2bd352ce63e7d9 is joining an existing cluster
I20260811 05:13:12.712311 8937 master_runner.cc:432] Initiating AddMaster RPC to add 127.28.143.189:40933
I20260811 05:13:12.719858 8097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:12.713411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38122 (local address 127.28.143.190:39435)
0811 05:13:12.713460 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:12.713463 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:12.713600 (+ 137us) server_negotiation.cc:407] Connection header received
0811 05:13:12.713714 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:12.713715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:12.713727 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:12.713774 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:12.714192 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.714986 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.715791 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.715985 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.716048 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:12.717217 (+ 1169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:12.717219 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:12.717219 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:12.717227 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:12.718663 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:12.719008 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:12.719009 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:12.719009 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:12.719071 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:12.719227 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:12.719227 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:12.719228 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:12.719354 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:12.719423 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:12.719681 (+ 258us) server_negotiation.cc:299] Negotiation successful
0811 05:13:12.719764 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 05:13:12.719877 9131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:12.713354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.0.0.1:38122)
0811 05:13:12.713535 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:12.713547 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:12.713621 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:12.713818 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:12.713818 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:12.714044 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:12.714104 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:12.714105 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.715089 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.715090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:12.715695 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:12.715697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.716124 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.716126 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:12.716193 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:12.716737 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:12.716742 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:12.717127 (+ 385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:12.718728 (+ 1601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:12.718728 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:12.718729 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:12.718945 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:12.719122 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:12.719123 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:12.719123 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:12.719177 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:12.719426 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:12.719427 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:12.719478 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 05:13:12.719631 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 05:13:12.719745 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":94,"threads_started":1}
I20260811 05:13:12.721796 9135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:12.713410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.189:40933 (local address 127.0.0.1:42434)
0811 05:13:12.713535 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:12.713547 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:12.713599 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:12.713737 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:12.713738 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:12.713998 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:12.714102 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:12.714104 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.716024 (+ 1920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.716025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:12.716729 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:12.716732 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.717407 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.717408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:12.717481 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:12.717811 (+ 330us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:12.717815 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:12.718282 (+ 467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:12.720592 (+ 2310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:12.720593 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:12.720593 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:12.720837 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:12.721066 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:12.721067 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:12.721067 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:12.721145 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:12.721479 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:12.721479 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:12.721511 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 05:13:12.721561 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 05:13:12.721671 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":93,"threads_started":1}
I20260811 05:13:12.721800 9087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:12.713582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42434 (local address 127.28.143.189:40933)
0811 05:13:12.713621 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:12.713623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:12.713632 (+ 9us) server_negotiation.cc:407] Connection header received
0811 05:13:12.713654 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:12.713655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:12.713664 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:12.713730 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:12.715099 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.715940 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.716813 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.717136 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.717223 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:12.718381 (+ 1158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:12.718382 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:12.718383 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:12.718390 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:12.720110 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:12.720938 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:12.720939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:12.720940 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:12.721000 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:12.721227 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:12.721228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:12.721228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:12.721355 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:12.721456 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:12.721600 (+ 144us) server_negotiation.cc:299] Negotiation successful
0811 05:13:12.721725 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":84,"server-negotiator.queue_time_us":19}
Aug 11 05:13:12 dist-test-slave-f7th krb5kdc[7922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425191, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.143.190@KRBTEST.COM for oryx/127.28.143.189@KRBTEST.COM
I20260811 05:13:12.729674 9087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:12.722837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42442 (local address 127.28.143.189:40933)
0811 05:13:12.722871 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:12.722872 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:12.723037 (+ 165us) server_negotiation.cc:407] Connection header received
0811 05:13:12.723112 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:12.723113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:12.723123 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:12.723168 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:12.723578 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.724028 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.724827 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.725013 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.725065 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:12.727279 (+ 2214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:12.727281 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:12.727281 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:12.727288 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:12.728475 (+ 1187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:12.728933 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:12.728934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:12.728934 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:12.728980 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:12.729193 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:12.729194 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:12.729195 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:12.729306 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:12.729355 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:12.729549 (+ 194us) server_negotiation.cc:299] Negotiation successful
0811 05:13:12.729610 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260811 05:13:12.729768 9136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:12.722802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.189:40933 (local address 127.0.0.1:42442)
0811 05:13:12.722944 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:12.722979 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:12.723049 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:12.723171 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:12.723175 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:12.723454 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:12.723506 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:12.723508 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.724094 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.724095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:12.724750 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:12.724753 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.725071 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.725072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:12.725157 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:12.725664 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:12.725669 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:12.727187 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:12.728554 (+ 1367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:12.728562 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:12.728565 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:12.728867 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:12.729056 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:12.729057 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:12.729057 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:12.729125 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:12.729361 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:12.729368 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:12.729429 (+ 61us) client_negotiation.cc:769] Sending connection context
0811 05:13:12.729520 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 05:13:12.729648 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":88,"threads_started":1}
I20260811 05:13:12.730425 8010 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.28.143.189:40933: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9"
member_type: NON_VOTER
last_known_addr {
host: "127.28.143.189"
port: 40933
}
attrs {
promote: true
}
}
dest_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b"
cas_config_opid_index: -1
I20260811 05:13:12.730872 8010 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: NON_VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: true } }
I20260811 05:13:12.731463 9139 raft_consensus.cc:2932] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER db647a2f54b34f5f8f2bd352ce63e7d9 (127.28.143.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: NON_VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: true } } }
I20260811 05:13:12.731884 9022 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:12.732654 9139 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.28.143.189:40933
I20260811 05:13:12.732702 9022 raft_consensus.cc:1261] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [term 1 FOLLOWER]: Refusing update from remote peer dc6e53c73bdd4fe18d564e13636cdd5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260811 05:13:12.732892 9137 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: NON_VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: true } } }
I20260811 05:13:12.733062 9137 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:12.733086 8937 master.cc:537] Master@127.28.143.189:40933 shutting down...
I20260811 05:13:12.733518 9137 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: NON_VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 05:13:12.734333 7983 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260811 05:13:12.735277 7983 consensus_peers.cc:487] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b -> Peer db647a2f54b34f5f8f2bd352ce63e7d9 (127.28.143.189:40933): Couldn't send request to peer db647a2f54b34f5f8f2bd352ce63e7d9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 05:13:12.736227 8937 raft_consensus.cc:2229] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:12.736321 8937 raft_consensus.cc:2258] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:12.736377 8937 tablet_replica.cc:330] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9: stopping tablet replica
I20260811 05:13:12.749941 8937 master.cc:552] Master@127.28.143.189:40933 shutdown complete.
I20260811 05:13:12.750000 8937 master_runner.cc:305] Clearing existing system tablet
I20260811 05:13:12.750119 8937 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:12.750170 8937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:12.750768 8937 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.002s sys 0.000s
I20260811 05:13:12.751251 9143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:12.751435 8937 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:12.751502 8937 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:12.751528 8937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-1/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-1/wal
uuid: "db647a2f54b34f5f8f2bd352ce63e7d9"
format_stamp: "Formatted at 2026-08-11 05:13:12 on dist-test-slave-f7th"
server_key: "28c94de5a92ec13f00acceb7d4799029"
server_key_iv: "6dc3cfdb2d98b2ec3cde12c003f1e126"
server_key_version: "encryptionkey@0"
I20260811 05:13:12.752494 8937 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:12.754312 8937 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 05:13:12.754406 8937 log.cc:1196] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 05:13:12.754606 8937 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9: Deleting consensus metadata
I20260811 05:13:12.754724 8937 master_runner.cc:315] Copying system tablet from 127.28.143.190:39435
I20260811 05:13:12.755497 8937 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9: tablet copy: Beginning tablet copy session from remote peer at address 127.28.143.190:39435
I20260811 05:13:12.761734 9151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:12.755814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.0.0.1:38128)
0811 05:13:12.756014 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:12.756023 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:12.756085 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:12.756216 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:12.756217 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:12.756413 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:12.756472 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:12.756474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.757248 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.757249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:12.757854 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:12.757856 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.758242 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.758244 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:12.758315 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:12.758849 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:12.758856 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:12.759171 (+ 315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:12.760732 (+ 1561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:12.760733 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:12.760734 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:12.760932 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:12.761123 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:12.761124 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:12.761125 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:12.761187 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:12.761428 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:12.761430 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:12.761478 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 05:13:12.761550 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 05:13:12.761649 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":98,"threads_started":1}
I20260811 05:13:12.761812 8097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:12.755907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38128 (local address 127.28.143.190:39435)
0811 05:13:12.755962 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:12.755964 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:12.756084 (+ 120us) server_negotiation.cc:407] Connection header received
0811 05:13:12.756148 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:12.756149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:12.756159 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:12.756202 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:12.756544 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.757166 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.757949 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.758170 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.758223 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:12.759260 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:12.759262 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:12.759263 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:12.759271 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:12.760659 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:12.761002 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:12.761003 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:12.761003 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:12.761067 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:12.761237 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:12.761238 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:12.761239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:12.761357 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:12.761420 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:12.761638 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 05:13:12.761726 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 05:13:12.762197 8056 tablet_copy_service.cc:140] P dc6e53c73bdd4fe18d564e13636cdd5b: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer db647a2f54b34f5f8f2bd352ce63e7d9 ({username='oryx', principal='oryx/127.28.143.189@KRBTEST.COM'} at 127.0.0.1:38128)
I20260811 05:13:12.762355 8056 tablet_copy_service.cc:161] P dc6e53c73bdd4fe18d564e13636cdd5b: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer db647a2f54b34f5f8f2bd352ce63e7d9 at {username='oryx', principal='oryx/127.28.143.189@KRBTEST.COM'} at 127.0.0.1:38128: session id = db647a2f54b34f5f8f2bd352ce63e7d9-00000000000000000000000000000000
I20260811 05:13:12.763150 8056 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 05:13:12.763793 8937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 05:13:12.764941 8937 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9: tablet copy: Starting download of 0 data blocks...
I20260811 05:13:12.765077 8937 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9: tablet copy: Starting download of 1 WAL segments...
I20260811 05:13:12.766738 8937 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 05:13:12.768059 8056 tablet_copy_service.cc:342] P dc6e53c73bdd4fe18d564e13636cdd5b: Request end of tablet copy session db647a2f54b34f5f8f2bd352ce63e7d9-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.28.143.189@KRBTEST.COM'} at 127.0.0.1:38128
I20260811 05:13:12.768123 8056 tablet_copy_service.cc:433] P dc6e53c73bdd4fe18d564e13636cdd5b: ending tablet copy session db647a2f54b34f5f8f2bd352ce63e7d9-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer db647a2f54b34f5f8f2bd352ce63e7d9
I20260811 05:13:12.770924 8937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:12.772264 9152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:12.772284 9153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:12.773424 9155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:12.773669 8937 server_base.cc:1034] running on GCE node
I20260811 05:13:12.773756 8937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:12.773904 8937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:12.775058 8937 hybrid_clock.cc:648] HybridClock initialized: now 1786425192775046 us; error 26 us; skew 500 ppm
Aug 11 05:13:12 dist-test-slave-f7th krb5kdc[7922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425192, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.143.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:12.776410 8937 init.cc:377] Logged in from keytab as oryx/127.28.143.189@KRBTEST.COM (short username oryx)
I20260811 05:13:12.776710 8937 webserver.cc:460] Webserver started at http://127.28.143.189:37137/ using document root <none> and password file <none>
I20260811 05:13:12.776832 8937 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:12.776868 8937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:12.777472 8937 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260811 05:13:12.777987 9166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:12.778124 8937 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:12.778177 8937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-1/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-1/wal
uuid: "db647a2f54b34f5f8f2bd352ce63e7d9"
format_stamp: "Formatted at 2026-08-11 05:13:12 on dist-test-slave-f7th"
server_key: "28c94de5a92ec13f00acceb7d4799029"
server_key_iv: "6dc3cfdb2d98b2ec3cde12c003f1e126"
server_key_version: "encryptionkey@0"
I20260811 05:13:12.778236 8937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:12.787969 8937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:12.789814 8937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:12.792294 8937 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.189:40933
I20260811 05:13:12.792337 9261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.189:40933 every 8 connection(s)
I20260811 05:13:12.792644 8937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-1/data/info.pb
I20260811 05:13:12.793391 9263 sys_catalog.cc:263] Verifying existing consensus state
I20260811 05:13:12.793828 9263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9: Bootstrap starting.
I20260811 05:13:12.796599 9263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 05:13:12.796847 9263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9: Bootstrap complete.
I20260811 05:13:12.797043 9263 raft_consensus.cc:348] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: NON_VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: true } }
I20260811 05:13:12.797117 9263 raft_consensus.cc:729] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: db647a2f54b34f5f8f2bd352ce63e7d9, State: Initialized, Role: LEARNER
I20260811 05:13:12.797153 9263 consensus_queue.cc:260] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: NON_VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: true } }
I20260811 05:13:12.797375 9263 sys_catalog.cc:564] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:13:12.797362 9269 sys_catalog.cc:455] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: NON_VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: true } } }
I20260811 05:13:12.797587 9269 sys_catalog.cc:458] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [sys.catalog]: This master's current role is: LEARNER
I20260811 05:13:12.798941 9290 catalog_manager.cc:1259] Loaded cluster ID: bbcde4e7dda24824a0a9e147d6d1f4b1
I20260811 05:13:12.799045 9290 catalog_manager.cc:1552] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9: loading cluster ID for follower catalog manager: success
I20260811 05:13:12.799974 9290 catalog_manager.cc:1574] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9: acquiring CA information for follower catalog manager: success
I20260811 05:13:12.800253 9290 catalog_manager.cc:1602] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 05:13:12.810636 9291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:12.803137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42450 (local address 127.28.143.189:40933)
0811 05:13:12.803273 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:12.803275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:12.803287 (+ 12us) server_negotiation.cc:407] Connection header received
0811 05:13:12.803314 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:12.803315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:12.803327 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:12.803421 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:12.803946 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.805024 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.805828 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.806156 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.806236 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:12.807513 (+ 1277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:12.807515 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:12.807515 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:12.807524 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:12.809243 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:12.809667 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:12.809668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:12.809668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:12.809742 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:12.809907 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:12.809909 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:12.809909 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:12.810057 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:12.810165 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:12.810335 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 05:13:12.810506 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":60,"threads_started":1}
WARNING: no policy specified for oryx/127.28.143.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.143.188@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.143.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.188.keytab.
Entry for principal oryx/127.28.143.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.188.keytab.
I20260811 05:13:12.846463 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.188:44183
--webserver_interface=127.28.143.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.188.keytab
--principal=oryx/127.28.143.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:43653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:39435,127.28.143.189:40933,127.28.143.188:44183
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:12.944430 9297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:12.944726 9297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:12.944764 9297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:12.946348 9297 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:12.946427 9297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:12.946444 9297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:12.946455 9297 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:12.946467 9297 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:13:12.948310 9297 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:43653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:39435,127.28.143.189:40933,127.28.143.188:44183
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.188.keytab
--principal=oryx/127.28.143.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.188:44183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.28.143.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:12.948602 9297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:12.948889 9297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:12.954767 9308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:12.954766 9304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:12.955051 9297 server_base.cc:1034] running on GCE node
W20260811 05:13:12.955173 9303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:12.955523 9297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:12.955868 9297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:12.957046 9297 hybrid_clock.cc:648] HybridClock initialized: now 1786425192956993 us; error 67 us; skew 500 ppm
Aug 11 05:13:12 dist-test-slave-f7th krb5kdc[7922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425192, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.143.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:12.960132 9297 init.cc:377] Logged in from keytab as oryx/127.28.143.188@KRBTEST.COM (short username oryx)
I20260811 05:13:12.960806 9297 webserver.cc:460] Webserver started at http://127.28.143.188:44449/ using document root <none> and password file <none>
I20260811 05:13:12.961051 9297 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:12.961105 9297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:12.961256 9297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:12.962458 9297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-2/data/instance:
uuid: "de49b21914144ffc8824e6f82f4422c4"
format_stamp: "Formatted at 2026-08-11 05:13:12 on dist-test-slave-f7th"
server_key: "58544902330eb0e518468c3dca981988"
server_key_iv: "68e6171ee353da682ca96571ba13fa70"
server_key_version: "encryptionkey@0"
I20260811 05:13:12.962942 9297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-2/wal/instance:
uuid: "de49b21914144ffc8824e6f82f4422c4"
format_stamp: "Formatted at 2026-08-11 05:13:12 on dist-test-slave-f7th"
server_key: "58544902330eb0e518468c3dca981988"
server_key_iv: "68e6171ee353da682ca96571ba13fa70"
server_key_version: "encryptionkey@0"
I20260811 05:13:12.964715 9297 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260811 05:13:12.965847 9318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:12.966100 9297 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:12.966208 9297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-2/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-2/wal
uuid: "de49b21914144ffc8824e6f82f4422c4"
format_stamp: "Formatted at 2026-08-11 05:13:12 on dist-test-slave-f7th"
server_key: "58544902330eb0e518468c3dca981988"
server_key_iv: "68e6171ee353da682ca96571ba13fa70"
server_key_version: "encryptionkey@0"
I20260811 05:13:12.966276 9297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:12.988055 9297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:12.992794 9297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:12.992987 9297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:12.998454 9297 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.188:44183
I20260811 05:13:12.998533 9414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.188:44183 every 8 connection(s)
I20260811 05:13:12.998966 9297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-2/data/info.pb
I20260811 05:13:13.000142 9416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:13.001885 9416 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:13:13.003897 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 9297
Aug 11 05:13:13 dist-test-slave-f7th krb5kdc[7922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425192, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.143.188@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM
Aug 11 05:13:13 dist-test-slave-f7th krb5kdc[7922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425191, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.28.143.188@KRBTEST.COM
I20260811 05:13:13.012145 8097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.002786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38140 (local address 127.28.143.190:39435)
0811 05:13:13.002851 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:13.002854 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:13.002998 (+ 144us) server_negotiation.cc:407] Connection header received
0811 05:13:13.003475 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:13.003476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:13.003487 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:13.003530 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:13.004255 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.004967 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.005988 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.006189 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.006245 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.008882 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:13.008884 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:13.008884 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:13.008892 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:13.010682 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.011155 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.011157 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.011157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.011216 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.011447 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.011447 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.011448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.011570 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:13.011633 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:13.011955 (+ 322us) server_negotiation.cc:299] Negotiation successful
0811 05:13:13.012044 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 05:13:13.012370 9418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.002662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.0.0.1:38140)
0811 05:13:13.002862 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:13.002907 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:13.003369 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:13.003593 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:13.003596 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:13.003875 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:13.004153 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.004159 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.005061 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.005062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.005869 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.005872 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.006279 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.006281 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.006388 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.006982 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:13.006993 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:13.008746 (+ 1753us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:13.010756 (+ 2010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:13.010764 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:13.010767 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:13.011082 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:13.011272 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:13.011273 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:13.011273 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:13.011388 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:13.011630 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:13.011637 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:13.011783 (+ 146us) client_negotiation.cc:769] Sending connection context
0811 05:13:13.011922 (+ 139us) client_negotiation.cc:240] Negotiation successful
0811 05:13:13.012144 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":101,"threads_started":1}
I20260811 05:13:13.013010 9416 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.143.189" port: 40933 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:13:13.013278 9419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.004427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41684 (local address 127.28.143.188:44183)
0811 05:13:13.004651 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:13.004656 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:13.004679 (+ 23us) server_negotiation.cc:407] Connection header received
0811 05:13:13.004701 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:13.004703 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:13.004723 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:13.004800 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:13.005391 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.006122 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.006944 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.007394 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.007473 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.010240 (+ 2767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:13.010249 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:13.010249 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:13.010256 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:13.011912 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.012298 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.012303 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.012303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.012364 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.012709 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.012710 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.012710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.012888 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:13.013019 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:13.013097 (+ 78us) server_negotiation.cc:299] Negotiation successful
0811 05:13:13.013183 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":112,"threads_started":1}
Aug 11 05:13:13 dist-test-slave-f7th krb5kdc[7922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425192, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.143.188@KRBTEST.COM for oryx/127.28.143.189@KRBTEST.COM
I20260811 05:13:13.019981 9418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.013283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.189:40933 (local address 127.0.0.1:42466)
0811 05:13:13.013325 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:13.013337 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:13.013414 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:13.013626 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:13.013627 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:13.013795 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:13.013837 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.013839 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.014554 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.014555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.015088 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.015090 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.015448 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.015449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.015504 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.015948 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:13.015953 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:13.017380 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:13.018984 (+ 1604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:13.018985 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:13.018986 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:13.019173 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:13.019357 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:13.019358 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:13.019359 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:13.019422 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:13.019716 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:13.019717 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:13.019761 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 05:13:13.019823 (+ 62us) client_negotiation.cc:240] Negotiation successful
0811 05:13:13.019911 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260811 05:13:13.020210 9291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.013415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42466 (local address 127.28.143.189:40933)
0811 05:13:13.013480 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:13.013482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:13.013489 (+ 7us) server_negotiation.cc:407] Connection header received
0811 05:13:13.013516 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:13.013517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:13.013530 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:13.013589 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:13.013913 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.014479 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.015166 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.015365 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.015419 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.017496 (+ 2077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:13.017498 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:13.017498 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:13.017506 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:13.018897 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.019235 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.019236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.019236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.019285 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.019509 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.019510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.019511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.019642 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:13.019707 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:13.019968 (+ 261us) server_negotiation.cc:299] Negotiation successful
0811 05:13:13.020096 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 05:13:13.020726 9416 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.143.188" port: 44183 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 05:13:13 dist-test-slave-f7th krb5kdc[7922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425192, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.143.188@KRBTEST.COM for oryx/127.28.143.188@KRBTEST.COM
I20260811 05:13:13.026895 9418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.021001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:44183 (local address 127.0.0.1:41696)
0811 05:13:13.021076 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:13.021088 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:13.021134 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:13.021251 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:13.021252 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:13.021378 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:13.021409 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.021410 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.022005 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.022006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.022462 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.022463 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.022792 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.022792 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.022834 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.023249 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:13.023254 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:13.024491 (+ 1237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:13.025983 (+ 1492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:13.025984 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:13.025985 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:13.026133 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:13.026299 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:13.026300 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:13.026300 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:13.026347 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:13.026563 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:13.026564 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:13.026601 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 05:13:13.026660 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 05:13:13.026766 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 05:13:13.026911 9419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.021093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41696 (local address 127.28.143.188:44183)
0811 05:13:13.021122 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:13.021123 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:13.021131 (+ 8us) server_negotiation.cc:407] Connection header received
0811 05:13:13.021195 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:13.021196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:13.021207 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:13.021251 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:13.021469 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.021933 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.022537 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.022740 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.022792 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.024574 (+ 1782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:13.024576 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:13.024576 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:13.024584 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:13.025859 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.026195 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.026197 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.026197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.026248 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.026395 (+ 147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.026395 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.026396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.026511 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:13.026566 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:13.026685 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 05:13:13.026816 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":16}
I20260811 05:13:13.028502 9416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4: Bootstrap starting.
I20260811 05:13:13.029384 9416 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:13.029652 9416 log.cc:824] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:13.030468 9416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4: No bootstrap required, opened a new log
I20260811 05:13:13.031911 9416 raft_consensus.cc:348] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 40933 } } peers { permanent_uuid: "de49b21914144ffc8824e6f82f4422c4" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 44183 } }
I20260811 05:13:13.032080 9416 raft_consensus.cc:374] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:13.032105 9416 raft_consensus.cc:729] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de49b21914144ffc8824e6f82f4422c4, State: Initialized, Role: FOLLOWER
I20260811 05:13:13.032229 9416 consensus_queue.cc:260] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 40933 } } peers { permanent_uuid: "de49b21914144ffc8824e6f82f4422c4" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 44183 } }
I20260811 05:13:13.032562 9416 sys_catalog.cc:564] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:13:13.032526 9421 sys_catalog.cc:455] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 40933 } } peers { permanent_uuid: "de49b21914144ffc8824e6f82f4422c4" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 44183 } } }
I20260811 05:13:13.032653 9421 sys_catalog.cc:458] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 05:13:13.034392 9436 catalog_manager.cc:1558] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:13:13.034615 9436 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:13:13.036986 9297 master_runner.cc:428] Detected that this master de49b21914144ffc8824e6f82f4422c4 is joining an existing cluster
I20260811 05:13:13.037036 9297 master_runner.cc:432] Initiating AddMaster RPC to add 127.28.143.188:44183
I20260811 05:13:13.044507 9419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.038532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41700 (local address 127.28.143.188:44183)
0811 05:13:13.038567 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:13.038569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:13.038575 (+ 6us) server_negotiation.cc:407] Connection header received
0811 05:13:13.038588 (+ 13us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:13.038588 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:13.038594 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:13.038636 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:13.039115 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.039791 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.040886 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.041153 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.041189 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.042245 (+ 1056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:13.042246 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:13.042246 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:13.042253 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:13.043482 (+ 1229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.043788 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.043789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.043789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.043837 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.043994 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.043995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.043995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.044095 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:13.044151 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:13.044290 (+ 139us) server_negotiation.cc:299] Negotiation successful
0811 05:13:13.044382 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":19}
I20260811 05:13:13.044590 9453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.038189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:44183 (local address 127.0.0.1:41700)
0811 05:13:13.038285 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:13.038293 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:13.038329 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:13.038649 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:13.038650 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:13.038990 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:13.039046 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.039048 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.039881 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.039881 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.040800 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.040804 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.041268 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.041269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.041321 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.041782 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:13.041786 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:13.042148 (+ 362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:13.043558 (+ 1410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:13.043559 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:13.043560 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:13.043728 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:13.043889 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:13.043889 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:13.043890 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:13.043944 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:13.044148 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:13.044149 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:13.044203 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 05:13:13.044264 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 05:13:13.044453 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81,"mutex_wait_us":129,"thread_start_us":62,"threads_started":1}
I20260811 05:13:13.065006 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 8136
I20260811 05:13:13.065446 9291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.038306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42480 (local address 127.28.143.189:40933)
0811 05:13:13.038380 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:13.038382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:13.038389 (+ 7us) server_negotiation.cc:407] Connection header received
0811 05:13:13.038422 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:13.038423 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:13.038434 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:13.038486 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:13.038985 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.039779 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.041156 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.041424 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.041465 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.062123 (+ 20658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:13.062125 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:13.062126 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:13.062136 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:13.063898 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.064511 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.064513 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.064513 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.064582 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.064825 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.064826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.064827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.064992 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:13.065058 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:13.065259 (+ 201us) server_negotiation.cc:299] Negotiation successful
0811 05:13:13.065353 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 05:13:13.065585 8097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.038172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38150 (local address 127.28.143.190:39435)
0811 05:13:13.038226 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:13.038228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:13.038334 (+ 106us) server_negotiation.cc:407] Connection header received
0811 05:13:13.038421 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:13.038423 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:13.038434 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:13.038485 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:13.039147 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.039914 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.041013 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.041317 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.041361 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.062206 (+ 20845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:13.062208 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:13.062208 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:13.062218 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:13.064025 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.064494 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.064495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.064496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.064570 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.065056 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.065057 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.065057 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.065193 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:13.065236 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:13.065423 (+ 187us) server_negotiation.cc:299] Negotiation successful
0811 05:13:13.065500 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 05:13:13.065860 9450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.037965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.189:40933 (local address 127.0.0.1:42480)
0811 05:13:13.038121 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:13.038133 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:13.038186 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:13.038503 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:13.038504 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:13.038798 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:13.038880 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.038883 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.039873 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.039874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.041073 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.041075 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.041540 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.041541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.041585 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.061423 (+ 19838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:13.061428 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:13.061889 (+ 461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:13.064026 (+ 2137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:13.064028 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:13.064028 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:13.064401 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:13.064694 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:13.064695 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:13.064695 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:13.064759 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:13.065084 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:13.065085 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:13.065140 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 05:13:13.065587 (+ 447us) client_negotiation.cc:240] Negotiation successful
0811 05:13:13.065722 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":32,"thread_start_us":90,"threads_started":1}
I20260811 05:13:13.065910 9451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.037986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.0.0.1:38150)
0811 05:13:13.038274 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:13.038283 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:13.038359 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:13.038722 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:13.038723 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:13.038974 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:13.039043 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.039045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.040015 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.040016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.040905 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.040908 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.041465 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.041466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.041527 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.061374 (+ 19847us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:13.061382 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:13.062039 (+ 657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:13.064105 (+ 2066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:13.064107 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:13.064107 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:13.064404 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:13.064908 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:13.064909 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:13.064909 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:13.064966 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:13.065282 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:13.065283 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:13.065317 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 05:13:13.065667 (+ 350us) client_negotiation.cc:240] Negotiation successful
0811 05:13:13.065778 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":20,"thread_start_us":88,"threads_started":1}
I20260811 05:13:13.096063 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.129:39087
--local_ip_for_outbound_sockets=127.28.143.129
--tserver_master_addrs=127.28.143.190:39435
--webserver_port=42899
--webserver_interface=127.28.143.129
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.129.keytab
--principal=oryx/127.28.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:43653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.28.143.190:39435,127.28.143.189:40933
--tserver_master_addrs=127.28.143.190:39435,127.28.143.189:40933,127.28.143.188:44183 with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 05:13:13 dist-test-slave-f7th krb5kdc[7922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425191, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.143.190@KRBTEST.COM for oryx/127.28.143.188@KRBTEST.COM
I20260811 05:13:13.101161 9136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.093355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:44183 (local address 127.0.0.1:41702)
0811 05:13:13.093411 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:13.093422 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:13.093461 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:13.093615 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:13.093617 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:13.093850 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:13.093920 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.093922 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.094764 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.094765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.095613 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.095616 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.095995 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.095997 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.096070 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.096591 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:13.096596 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:13.098126 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:13.099937 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:13.099939 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:13.099939 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:13.100171 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:13.100392 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:13.100393 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:13.100393 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:13.100465 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:13.100789 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:13.100791 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:13.100846 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 05:13:13.100927 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 05:13:13.101048 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260811 05:13:13.101200 9419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.093426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41702 (local address 127.28.143.188:44183)
0811 05:13:13.093470 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:13.093472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:13.093486 (+ 14us) server_negotiation.cc:407] Connection header received
0811 05:13:13.093528 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:13.093529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:13.093541 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:13.093601 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:13.094010 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.094661 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.095713 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.095917 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.095977 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.098225 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:13.098227 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:13.098227 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:13.098237 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:13.099832 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.100257 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.100258 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.100258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.100324 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.100532 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.100533 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.100533 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.100716 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:13.100790 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:13.100961 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 05:13:13.101108 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 05:13:13.102114 8010 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.28.143.188:44183: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "de49b21914144ffc8824e6f82f4422c4"
member_type: NON_VOTER
last_known_addr {
host: "127.28.143.188"
port: 44183
}
attrs {
promote: true
}
}
dest_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b"
cas_config_opid_index: 5
I20260811 05:13:13.102408 8010 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: NON_VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: true } } peers { permanent_uuid: "de49b21914144ffc8824e6f82f4422c4" member_type: NON_VOTER last_known_addr { host: "127.28.143.188" port: 44183 } attrs { promote: true } }
I20260811 05:13:13.102860 9137 raft_consensus.cc:2932] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER de49b21914144ffc8824e6f82f4422c4 (127.28.143.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: NON_VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: true } } peers { permanent_uuid: "de49b21914144ffc8824e6f82f4422c4" member_type: NON_VOTER last_known_addr { host: "127.28.143.188" port: 44183 } attrs { promote: true } } }
I20260811 05:13:13.103322 9372 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:13.103914 9137 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.28.143.188:44183
I20260811 05:13:13.103993 9137 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: NON_VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: true } } peers { permanent_uuid: "de49b21914144ffc8824e6f82f4422c4" member_type: NON_VOTER last_known_addr { host: "127.28.143.188" port: 44183 } attrs { promote: true } } }
I20260811 05:13:13.104070 9137 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:13.104192 9372 raft_consensus.cc:1261] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [term 1 FOLLOWER]: Refusing update from remote peer dc6e53c73bdd4fe18d564e13636cdd5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 05:13:13.104561 9297 master.cc:537] Master@127.28.143.188:44183 shutting down...
I20260811 05:13:13.104554 9139 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de49b21914144ffc8824e6f82f4422c4" member_type: NON_VOTER last_known_addr { host: "127.28.143.188" port: 44183 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 05:13:13.105990 7983 consensus_peers.cc:487] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b -> Peer de49b21914144ffc8824e6f82f4422c4 (127.28.143.188:44183): Couldn't send request to peer de49b21914144ffc8824e6f82f4422c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 05:13:13.108608 9297 raft_consensus.cc:2229] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:13.108755 9297 raft_consensus.cc:2258] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:13.108805 9297 tablet_replica.cc:330] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4: stopping tablet replica
I20260811 05:13:13.110657 9291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.103050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42488 (local address 127.28.143.189:40933)
0811 05:13:13.106321 (+ 3271us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:13.106324 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:13.106475 (+ 151us) server_negotiation.cc:407] Connection header received
0811 05:13:13.106503 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:13.106504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:13.106529 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:13.106577 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:13.106767 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.107605 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.109268 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.109903 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.109984 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.110064 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:13.110539 (+ 475us) server_negotiation.cc:299] Negotiation successful
0811 05:13:13.110593 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3225}
I20260811 05:13:13.110376 9136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.102937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.189:40933 (local address 127.0.0.1:42488)
0811 05:13:13.106362 (+ 3425us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:13.106374 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:13.106428 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:13.106644 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:13.106645 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:13.106647 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:13.106697 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.106699 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.107702 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.107704 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.109158 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.109163 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.110103 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.110104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.110176 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.110203 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 05:13:13.110251 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 05:13:13.110298 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3405}
I20260811 05:13:13.111456 9211 raft_consensus.cc:1261] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [term 1 LEARNER]: Refusing update from remote peer dc6e53c73bdd4fe18d564e13636cdd5b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 05:13:13.111694 9139 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: NON_VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 05:13:13.112424 9211 raft_consensus.cc:2932] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER de49b21914144ffc8824e6f82f4422c4 (127.28.143.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: NON_VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: true } } peers { permanent_uuid: "de49b21914144ffc8824e6f82f4422c4" member_type: NON_VOTER last_known_addr { host: "127.28.143.188" port: 44183 } attrs { promote: true } } }
I20260811 05:13:13.113830 9269 sys_catalog.cc:455] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc6e53c73bdd4fe18d564e13636cdd5b. Latest consensus state: current_term: 1 leader_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: NON_VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: true } } peers { permanent_uuid: "de49b21914144ffc8824e6f82f4422c4" member_type: NON_VOTER last_known_addr { host: "127.28.143.188" port: 44183 } attrs { promote: true } } }
I20260811 05:13:13.113924 9269 sys_catalog.cc:458] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [sys.catalog]: This master's current role is: LEARNER
I20260811 05:13:13.113956 9269 sys_catalog.cc:455] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: NON_VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: true } } peers { permanent_uuid: "de49b21914144ffc8824e6f82f4422c4" member_type: NON_VOTER last_known_addr { host: "127.28.143.188" port: 44183 } attrs { promote: true } } }
I20260811 05:13:13.113999 9269 sys_catalog.cc:458] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [sys.catalog]: This master's current role is: LEARNER
I20260811 05:13:13.114250 9139 raft_consensus.cc:1050] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b: attempting to promote NON_VOTER db647a2f54b34f5f8f2bd352ce63e7d9 to VOTER
I20260811 05:13:13.114461 9139 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: false } } peers { permanent_uuid: "de49b21914144ffc8824e6f82f4422c4" member_type: NON_VOTER last_known_addr { host: "127.28.143.188" port: 44183 } attrs { promote: true } }
I20260811 05:13:13.115164 9211 raft_consensus.cc:1261] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [term 1 LEARNER]: Refusing update from remote peer dc6e53c73bdd4fe18d564e13636cdd5b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 05:13:13.115345 9137 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 05:13:13.115782 9136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.114986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:44183 (local address 127.0.0.1:41704)
0811 05:13:13.115700 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:13.115752 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.188:44183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":694}
I20260811 05:13:13.116058 9137 raft_consensus.cc:2932] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, db647a2f54b34f5f8f2bd352ce63e7d9 (127.28.143.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: false } } peers { permanent_uuid: "de49b21914144ffc8824e6f82f4422c4" member_type: NON_VOTER last_known_addr { host: "127.28.143.188" port: 44183 } attrs { promote: true } } }
I20260811 05:13:13.116191 9136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.116079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:44183 (local address 127.0.0.1:41720)
0811 05:13:13.116146 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:13.116168 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.188:44183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 05:13:13.116353 9211 raft_consensus.cc:2932] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, db647a2f54b34f5f8f2bd352ce63e7d9 (127.28.143.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: false } } peers { permanent_uuid: "de49b21914144ffc8824e6f82f4422c4" member_type: NON_VOTER last_known_addr { host: "127.28.143.188" port: 44183 } attrs { promote: true } } }
I20260811 05:13:13.117139 9137 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: false } } peers { permanent_uuid: "de49b21914144ffc8824e6f82f4422c4" member_type: NON_VOTER last_known_addr { host: "127.28.143.188" port: 44183 } attrs { promote: true } } }
I20260811 05:13:13.117215 9137 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:13.117257 9137 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: false } } peers { permanent_uuid: "de49b21914144ffc8824e6f82f4422c4" member_type: NON_VOTER last_known_addr { host: "127.28.143.188" port: 44183 } attrs { promote: true } } }
I20260811 05:13:13.117293 9137 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [sys.catalog]: This master's current role is: LEADER
W20260811 05:13:13.117377 7983 consensus_peers.cc:487] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b -> Peer de49b21914144ffc8824e6f82f4422c4 (127.28.143.188:44183): Couldn't send request to peer de49b21914144ffc8824e6f82f4422c4. Status: Network error: Client connection negotiation failed: client connection to 127.28.143.188:44183: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 05:13:13.117503 9460 sys_catalog.cc:455] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: false } } peers { permanent_uuid: "de49b21914144ffc8824e6f82f4422c4" member_type: NON_VOTER last_known_addr { host: "127.28.143.188" port: 44183 } attrs { promote: true } } }
I20260811 05:13:13.117586 9460 sys_catalog.cc:458] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:13:13.122051 9297 master.cc:552] Master@127.28.143.188:44183 shutdown complete.
I20260811 05:13:13.122113 9297 master_runner.cc:305] Clearing existing system tablet
I20260811 05:13:13.122246 9297 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:13.122292 9297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:13.123051 9297 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:13.123687 9474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:13.123931 9297 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:13.124003 9297 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:13.124028 9297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-2/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-2/wal
uuid: "de49b21914144ffc8824e6f82f4422c4"
format_stamp: "Formatted at 2026-08-11 05:13:12 on dist-test-slave-f7th"
server_key: "58544902330eb0e518468c3dca981988"
server_key_iv: "68e6171ee353da682ca96571ba13fa70"
server_key_version: "encryptionkey@0"
I20260811 05:13:13.125012 9297 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:13.126765 9297 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 05:13:13.126894 9297 log.cc:1196] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 05:13:13.127177 9297 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4: Deleting consensus metadata
I20260811 05:13:13.127298 9297 master_runner.cc:315] Copying system tablet from 127.28.143.190:39435
I20260811 05:13:13.127940 9297 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4: tablet copy: Beginning tablet copy session from remote peer at address 127.28.143.190:39435
I20260811 05:13:13.135605 9481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.128279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.0.0.1:38160)
0811 05:13:13.128419 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:13.128430 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:13.128500 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:13.128689 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:13.128690 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:13.128928 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:13.128994 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.128997 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.129949 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.129950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.130799 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.130802 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.131269 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.131271 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.131347 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.131866 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:13.131872 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:13.132423 (+ 551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:13.134358 (+ 1935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:13.134360 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:13.134360 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:13.134595 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:13.134831 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:13.134832 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:13.134832 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:13.134917 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:13.135213 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:13.135214 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:13.135283 (+ 69us) client_negotiation.cc:769] Sending connection context
0811 05:13:13.135363 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 05:13:13.135488 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":74,"threads_started":1}
I20260811 05:13:13.135704 8097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.128353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38160 (local address 127.28.143.190:39435)
0811 05:13:13.128409 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:13.128412 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:13.128484 (+ 72us) server_negotiation.cc:407] Connection header received
0811 05:13:13.128558 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:13.128559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:13.128570 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:13.128671 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:13.129083 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.129843 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.130894 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.131185 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.131252 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.132515 (+ 1263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:13.132516 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:13.132517 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:13.132526 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:13.134285 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.134685 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.134687 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.134687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.134758 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.134987 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.134988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.134988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.135136 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:13.135210 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:13.135398 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 05:13:13.135614 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 05:13:13.136291 8056 tablet_copy_service.cc:140] P dc6e53c73bdd4fe18d564e13636cdd5b: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer de49b21914144ffc8824e6f82f4422c4 ({username='oryx', principal='oryx/127.28.143.188@KRBTEST.COM'} at 127.0.0.1:38160)
I20260811 05:13:13.136360 8056 tablet_copy_service.cc:161] P dc6e53c73bdd4fe18d564e13636cdd5b: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer de49b21914144ffc8824e6f82f4422c4 at {username='oryx', principal='oryx/127.28.143.188@KRBTEST.COM'} at 127.0.0.1:38160: session id = de49b21914144ffc8824e6f82f4422c4-00000000000000000000000000000000
I20260811 05:13:13.137152 8056 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 05:13:13.137739 9297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 05:13:13.139165 9297 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4: tablet copy: Starting download of 0 data blocks...
I20260811 05:13:13.139369 9297 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4: tablet copy: Starting download of 1 WAL segments...
I20260811 05:13:13.140273 9297 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 05:13:13.141350 8056 tablet_copy_service.cc:342] P dc6e53c73bdd4fe18d564e13636cdd5b: Request end of tablet copy session de49b21914144ffc8824e6f82f4422c4-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.28.143.188@KRBTEST.COM'} at 127.0.0.1:38160
I20260811 05:13:13.141405 8056 tablet_copy_service.cc:433] P dc6e53c73bdd4fe18d564e13636cdd5b: ending tablet copy session de49b21914144ffc8824e6f82f4422c4-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer de49b21914144ffc8824e6f82f4422c4
I20260811 05:13:13.144981 9297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:13.146386 9484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:13.149669 9485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:13.149822 9487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:13.150071 9297 server_base.cc:1034] running on GCE node
I20260811 05:13:13.150168 9297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:13.150310 9297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:13.153489 9297 hybrid_clock.cc:648] HybridClock initialized: now 1786425193153479 us; error 35 us; skew 500 ppm
Aug 11 05:13:13 dist-test-slave-f7th krb5kdc[7922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425193, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.143.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:13.155157 9297 init.cc:377] Logged in from keytab as oryx/127.28.143.188@KRBTEST.COM (short username oryx)
I20260811 05:13:13.155464 9297 webserver.cc:460] Webserver started at http://127.28.143.188:44481/ using document root <none> and password file <none>
I20260811 05:13:13.155743 9297 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:13.155782 9297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:13.156541 9297 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:13.157254 9500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:13.157373 9297 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 05:13:13.157414 9297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-2/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-2/wal
uuid: "de49b21914144ffc8824e6f82f4422c4"
format_stamp: "Formatted at 2026-08-11 05:13:12 on dist-test-slave-f7th"
server_key: "58544902330eb0e518468c3dca981988"
server_key_iv: "68e6171ee353da682ca96571ba13fa70"
server_key_version: "encryptionkey@0"
I20260811 05:13:13.157472 9297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:13.185930 9297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:13.187879 9297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:13.191512 9297 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.188:44183
I20260811 05:13:13.191990 9297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/master-2/data/info.pb
I20260811 05:13:13.192713 9590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.188:44183 every 8 connection(s)
I20260811 05:13:13.193076 9595 sys_catalog.cc:263] Verifying existing consensus state
I20260811 05:13:13.193576 9595 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4: Bootstrap starting.
I20260811 05:13:13.196904 9595 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 05:13:13.197213 9595 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4: Bootstrap complete.
I20260811 05:13:13.197460 9595 raft_consensus.cc:348] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: false } } peers { permanent_uuid: "de49b21914144ffc8824e6f82f4422c4" member_type: NON_VOTER last_known_addr { host: "127.28.143.188" port: 44183 } attrs { promote: true } }
I20260811 05:13:13.197556 9595 raft_consensus.cc:729] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: de49b21914144ffc8824e6f82f4422c4, State: Initialized, Role: LEARNER
I20260811 05:13:13.197595 9595 consensus_queue.cc:260] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: false } } peers { permanent_uuid: "de49b21914144ffc8824e6f82f4422c4" member_type: NON_VOTER last_known_addr { host: "127.28.143.188" port: 44183 } attrs { promote: true } }
I20260811 05:13:13.197794 9595 sys_catalog.cc:564] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:13:13.197829 9603 sys_catalog.cc:455] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: false } } peers { permanent_uuid: "de49b21914144ffc8824e6f82f4422c4" member_type: NON_VOTER last_known_addr { host: "127.28.143.188" port: 44183 } attrs { promote: true } } }
I20260811 05:13:13.198079 9603 sys_catalog.cc:458] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [sys.catalog]: This master's current role is: LEARNER
I20260811 05:13:13.199369 9624 catalog_manager.cc:1259] Loaded cluster ID: bbcde4e7dda24824a0a9e147d6d1f4b1
I20260811 05:13:13.199477 9624 catalog_manager.cc:1552] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4: loading cluster ID for follower catalog manager: success
I20260811 05:13:13.200534 9624 catalog_manager.cc:1574] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4: acquiring CA information for follower catalog manager: success
I20260811 05:13:13.200862 9624 catalog_manager.cc:1602] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 05:13:13.203040 9454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:13.203238 9454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:13.203274 9454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:13.204468 9454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:13.204516 9454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:13.204557 9454 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.129
I20260811 05:13:13.206133 9454 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:43653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.129.keytab
--principal=oryx/127.28.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.129:39087
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.143.129
--webserver_port=42899
--tserver_master_addrs=127.28.143.190:39435,127.28.143.189:40933,127.28.143.188:44183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.129
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:13.206383 9454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:13.206614 9454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:13.211911 9635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:13.212034 9454 server_base.cc:1034] running on GCE node
W20260811 05:13:13.211931 9638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:13.211936 9634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:13.212409 9454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:13.212702 9454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:13.213883 9454 hybrid_clock.cc:648] HybridClock initialized: now 1786425193213868 us; error 31 us; skew 500 ppm
Aug 11 05:13:13 dist-test-slave-f7th krb5kdc[7922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425193, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:13.216714 9454 init.cc:377] Logged in from keytab as oryx/127.28.143.129@KRBTEST.COM (short username oryx)
I20260811 05:13:13.217235 9454 webserver.cc:460] Webserver started at http://127.28.143.129:42899/ using document root <none> and password file <none>
I20260811 05:13:13.217460 9454 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:13.217537 9454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:13.219038 9454 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:13.219813 9653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:13.220116 9454 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:13.220233 9454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-0/wal
uuid: "4c4cc9aaf7b14d99920b5b6edca2a68a"
format_stamp: "Formatted at 2026-08-11 05:13:12 on dist-test-slave-f7th"
server_key: "4cd0fd5c1783b66a8af65c0baa84897d"
server_key_iv: "d7c3e8930a29ba11b1f0f7a0ae4b18ac"
server_key_version: "encryptionkey@0"
I20260811 05:13:13.220659 9454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:13.253921 9454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:13.256485 9454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:13.256675 9454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:13.256938 9454 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:13.257283 9454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:13.257321 9454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:13.257354 9454 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:13.257378 9454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:13.263778 9454 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.129:39087
I20260811 05:13:13.263823 9878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.129:39087 every 8 connection(s)
I20260811 05:13:13.264184 9454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
I20260811 05:13:13.273715 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 9454
I20260811 05:13:13.273890 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 8414
Aug 11 05:13:13 dist-test-slave-f7th krb5kdc[7922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425193, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.143.129@KRBTEST.COM for oryx/127.28.143.188@KRBTEST.COM
Aug 11 05:13:13 dist-test-slave-f7th krb5kdc[7922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425193, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.143.129@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM
Aug 11 05:13:13 dist-test-slave-f7th krb5kdc[7922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425193, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.143.129@KRBTEST.COM for oryx/127.28.143.189@KRBTEST.COM
I20260811 05:13:13.282532 8097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.265067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:53687 (local address 127.28.143.190:39435)
0811 05:13:13.265128 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:13.265130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:13.265396 (+ 266us) server_negotiation.cc:407] Connection header received
0811 05:13:13.266185 (+ 789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:13.266187 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:13.266199 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:13.266246 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:13.267079 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.267856 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.269239 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.269529 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.269589 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.278006 (+ 8417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:13.278009 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:13.278009 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:13.278020 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:13.280928 (+ 2908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.281412 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.281414 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.281415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.281480 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.281732 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.281733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.281733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.281865 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:13.281936 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:13.282334 (+ 398us) server_negotiation.cc:299] Negotiation successful
0811 05:13:13.282434 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 05:13:13.283146 9888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.265002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.28.143.129:53687)
0811 05:13:13.265213 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:13.265259 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:13.265860 (+ 601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:13.266278 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:13.266284 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:13.266732 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:13.266979 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.266985 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.267931 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.267932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.269152 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.269156 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.269640 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.269641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.269707 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.273353 (+ 3646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:13.273367 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:13.277856 (+ 4489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:13.281015 (+ 3159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:13.281029 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:13.281032 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:13.281333 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:13.281542 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:13.281543 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:13.281544 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:13.281672 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:13.281965 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:13.281973 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:13.282156 (+ 183us) client_negotiation.cc:769] Sending connection context
0811 05:13:13.282648 (+ 492us) client_negotiation.cc:240] Negotiation successful
0811 05:13:13.282863 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":143,"thread_start_us":92,"threads_started":1}
I20260811 05:13:13.283705 9882 heartbeater.cc:347] Connected to a master server at 127.28.143.190:39435
I20260811 05:13:13.283794 9882 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:13.283957 9882 heartbeater.cc:511] Master 127.28.143.190:39435 requested a full tablet report, sending...
I20260811 05:13:13.284390 8010 ts_manager.cc:194] Re-registered known tserver with Master: 4c4cc9aaf7b14d99920b5b6edca2a68a (127.28.143.129:39087)
I20260811 05:13:13.284936 8010 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.143.129@KRBTEST.COM'} at 127.28.143.129:53687
I20260811 05:13:13.285208 9291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.265395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:35091 (local address 127.28.143.189:40933)
0811 05:13:13.265443 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:13.265445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:13.265478 (+ 33us) server_negotiation.cc:407] Connection header received
0811 05:13:13.266050 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:13.266051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:13.266069 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:13.266122 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:13.267105 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.267855 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.269127 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.269384 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.269454 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.280930 (+ 11476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:13.280933 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:13.280933 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:13.280943 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:13.283333 (+ 2390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.284331 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.284333 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.284333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.284399 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.284654 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.284655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.284656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.284797 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:13.284839 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:13.285033 (+ 194us) server_negotiation.cc:299] Negotiation successful
0811 05:13:13.285125 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 05:13:13.286046 9890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.265267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.189:40933 (local address 127.28.143.129:35091)
0811 05:13:13.265420 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:13.265430 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:13.265910 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:13.266501 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:13.266510 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:13.266729 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:13.266980 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.266987 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.267974 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.267975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.269035 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.269038 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.269472 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.269472 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.269616 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.273351 (+ 3735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:13.273365 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:13.280776 (+ 7411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:13.283507 (+ 2731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:13.283508 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:13.283508 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:13.284243 (+ 735us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:13.284490 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:13.284490 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:13.284491 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:13.284549 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:13.284882 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:13.284883 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:13.284926 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 05:13:13.285853 (+ 927us) client_negotiation.cc:240] Negotiation successful
0811 05:13:13.285966 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":137,"thread_start_us":85,"threads_started":1}
I20260811 05:13:13.293367 9883 heartbeater.cc:347] Connected to a master server at 127.28.143.189:40933
I20260811 05:13:13.293448 9883 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:13.293572 9883 heartbeater.cc:511] Master 127.28.143.189:40933 requested a full tablet report, sending...
I20260811 05:13:13.293983 9194 ts_manager.cc:194] Registered new tserver with Master: 4c4cc9aaf7b14d99920b5b6edca2a68a (127.28.143.129:39087)
I20260811 05:13:13.295382 9889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.264954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:44183 (local address 127.28.143.129:54109)
0811 05:13:13.265213 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:13.265259 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:13.265904 (+ 645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:13.266441 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:13.266442 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:13.266736 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:13.266980 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.266987 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.267890 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.267891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.269155 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.269157 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.269577 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.269578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.269645 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.273358 (+ 3713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:13.273367 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:13.275332 (+ 1965us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:13.294312 (+ 18980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:13.294313 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:13.294314 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:13.294524 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:13.294759 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:13.294759 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:13.294760 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:13.294807 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:13.295076 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:13.295076 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:13.295119 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 05:13:13.295166 (+ 47us) client_negotiation.cc:240] Negotiation successful
0811 05:13:13.295274 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":91,"thread_start_us":87,"threads_started":1}
I20260811 05:13:13.295409 9886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.264995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:54109 (local address 127.28.143.188:44183)
0811 05:13:13.265249 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:13.265252 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:13.265449 (+ 197us) server_negotiation.cc:407] Connection header received
0811 05:13:13.266304 (+ 855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:13.266305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:13.266318 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:13.266411 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:13.267091 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.267760 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.269258 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.269488 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.269567 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.275474 (+ 5907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:13.275476 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:13.275476 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:13.275486 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:13.294222 (+ 18736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.294608 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.294609 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.294609 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.294691 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.294859 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.294859 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.294860 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.295011 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:13.295079 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:13.295205 (+ 126us) server_negotiation.cc:299] Negotiation successful
0811 05:13:13.295320 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":62,"threads_started":1}
I20260811 05:13:13.295894 9879 heartbeater.cc:347] Connected to a master server at 127.28.143.188:44183
I20260811 05:13:13.295953 9879 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:13.296012 9879 heartbeater.cc:511] Master 127.28.143.188:44183 requested a full tablet report, sending...
I20260811 05:13:13.296386 9526 ts_manager.cc:194] Registered new tserver with Master: 4c4cc9aaf7b14d99920b5b6edca2a68a (127.28.143.129:39087)
I20260811 05:13:13.297397 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.130:34837
--local_ip_for_outbound_sockets=127.28.143.130
--tserver_master_addrs=127.28.143.190:39435
--webserver_port=41741
--webserver_interface=127.28.143.130
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.130.keytab
--principal=oryx/127.28.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:43653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.28.143.190:39435,127.28.143.189:40933
--tserver_master_addrs=127.28.143.190:39435,127.28.143.189:40933,127.28.143.188:44183 with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:13.387013 9893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:13.387264 9893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:13.387290 9893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:13.388859 9893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:13.388932 9893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:13.388965 9893 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.130
I20260811 05:13:13.390761 9893 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:43653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.130.keytab
--principal=oryx/127.28.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.130:34837
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.143.130
--webserver_port=41741
--tserver_master_addrs=127.28.143.190:39435,127.28.143.189:40933,127.28.143.188:44183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.130
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:13.391067 9893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:13.391332 9893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:13.397150 9904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:13.397113 9902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:13.397325 9893 server_base.cc:1034] running on GCE node
W20260811 05:13:13.397127 9909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:13.397794 9893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:13.398093 9893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:13.399308 9893 hybrid_clock.cc:648] HybridClock initialized: now 1786425193399303 us; error 50 us; skew 500 ppm
Aug 11 05:13:13 dist-test-slave-f7th krb5kdc[7922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425193, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:13.402285 9893 init.cc:377] Logged in from keytab as oryx/127.28.143.130@KRBTEST.COM (short username oryx)
I20260811 05:13:13.402881 9893 webserver.cc:460] Webserver started at http://127.28.143.130:41741/ using document root <none> and password file <none>
I20260811 05:13:13.403123 9893 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:13.403199 9893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:13.404793 9893 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 05:13:13.405709 9924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:13.405992 9893 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:13.406109 9893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-1/wal
uuid: "fda127cd3b28435498b97c4445c40438"
format_stamp: "Formatted at 2026-08-11 05:13:12 on dist-test-slave-f7th"
server_key: "cc8c2ee870d4644bd758134621804df8"
server_key_iv: "d65763378af5c9ca329b6536c113f218"
server_key_version: "encryptionkey@0"
I20260811 05:13:13.406497 9893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:13.425765 9893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:13.428215 9893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:13.428382 9893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:13.428671 9893 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:13.429009 9893 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:13.429044 9893 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:13.429075 9893 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:13.429093 9893 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:13.435854 9893 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.130:34837
I20260811 05:13:13.435932 10130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.130:34837 every 8 connection(s)
I20260811 05:13:13.436306 9893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
Aug 11 05:13:13 dist-test-slave-f7th krb5kdc[7922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425193, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.143.130@KRBTEST.COM for oryx/127.28.143.188@KRBTEST.COM
Aug 11 05:13:13 dist-test-slave-f7th krb5kdc[7922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425193, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.143.130@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM
Aug 11 05:13:13 dist-test-slave-f7th krb5kdc[7922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425193, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.143.130@KRBTEST.COM for oryx/127.28.143.189@KRBTEST.COM
I20260811 05:13:13.444193 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 9893
I20260811 05:13:13.444339 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 8680
I20260811 05:13:13.447845 9291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.437568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:54341 (local address 127.28.143.189:40933)
0811 05:13:13.437610 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:13.437612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:13.437622 (+ 10us) server_negotiation.cc:407] Connection header received
0811 05:13:13.438553 (+ 931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:13.438555 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:13.438568 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:13.438597 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:13.439384 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.440168 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.441324 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.441631 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.441670 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.444543 (+ 2873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:13.444545 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:13.444546 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:13.444555 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:13.446204 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.446703 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.446704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.446705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.446765 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.447031 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.447032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.447032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.447179 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:13.447222 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:13.447648 (+ 426us) server_negotiation.cc:299] Negotiation successful
0811 05:13:13.447739 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 05:13:13.447845 9886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.437144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:54233 (local address 127.28.143.188:44183)
0811 05:13:13.437194 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:13.437196 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:13.437473 (+ 277us) server_negotiation.cc:407] Connection header received
0811 05:13:13.438527 (+ 1054us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:13.438528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:13.438558 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:13.438588 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:13.439376 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.439882 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.440988 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.441202 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.441258 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.443721 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:13.443723 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:13.443723 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:13.443733 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:13.445850 (+ 2117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.446614 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.446615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.446616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.446699 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.447031 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.447032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.447032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.447179 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:13.447222 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:13.447638 (+ 416us) server_negotiation.cc:299] Negotiation successful
0811 05:13:13.447741 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 05:13:13.447862 8097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.437139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:60087 (local address 127.28.143.190:39435)
0811 05:13:13.437199 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:13.437201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:13.437447 (+ 246us) server_negotiation.cc:407] Connection header received
0811 05:13:13.438563 (+ 1116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:13.438564 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:13.438577 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:13.438627 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:13.439385 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.440168 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.441324 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.441630 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.441670 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.443926 (+ 2256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:13.443928 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:13.443928 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:13.443938 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:13.446051 (+ 2113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.446811 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.446812 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.446813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.446912 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.447176 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.447177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.447178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.447340 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:13.447390 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:13.447665 (+ 275us) server_negotiation.cc:299] Negotiation successful
0811 05:13:13.447769 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 05:13:13.448586 10140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.437476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.189:40933 (local address 127.28.143.130:54341)
0811 05:13:13.437583 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:13.437591 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:13.438414 (+ 823us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:13.438722 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:13.438723 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:13.439123 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:13.439276 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.439282 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.440265 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.440266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.441238 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.441240 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.441750 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.441751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.441824 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.442381 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:13.442385 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:13.444426 (+ 2041us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:13.446295 (+ 1869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:13.446296 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:13.446296 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:13.446616 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:13.446857 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:13.446857 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:13.446858 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:13.446949 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:13.447270 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:13.447278 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:13.447473 (+ 195us) client_negotiation.cc:769] Sending connection context
0811 05:13:13.448003 (+ 530us) client_negotiation.cc:240] Negotiation successful
0811 05:13:13.448359 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":165,"thread_start_us":59,"threads_started":1}
I20260811 05:13:13.448943 10139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.437146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:44183 (local address 127.28.143.130:54233)
0811 05:13:13.437327 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:13.437367 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:13.438390 (+ 1023us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:13.438718 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:13.438722 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:13.439062 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:13.439274 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.439279 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.439988 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.439989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.440907 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.440910 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.441275 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.441276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.441406 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.441927 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:13.441936 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:13.443584 (+ 1648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:13.445965 (+ 2381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:13.445978 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:13.445982 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:13.446503 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:13.446774 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:13.446775 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:13.446775 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:13.446950 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:13.447279 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:13.447279 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:13.447473 (+ 194us) client_negotiation.cc:769] Sending connection context
0811 05:13:13.448462 (+ 989us) client_negotiation.cc:240] Negotiation successful
0811 05:13:13.448558 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":4,"thread_start_us":93,"threads_started":1}
I20260811 05:13:13.448594 10138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.437145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.28.143.130:60087)
0811 05:13:13.437321 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:13.437367 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:13.438403 (+ 1036us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:13.438722 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:13.438723 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:13.439126 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:13.439276 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.439282 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.440265 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.440266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.441219 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.441222 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.441750 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.441751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.441824 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.442297 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:13.442301 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:13.443836 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:13.446162 (+ 2326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:13.446164 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:13.446164 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:13.446580 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:13.447006 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:13.447007 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:13.447007 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:13.447086 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:13.447445 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:13.447446 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:13.447500 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 05:13:13.448007 (+ 507us) client_negotiation.cc:240] Negotiation successful
0811 05:13:13.448279 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":87,"thread_start_us":109,"threads_started":1}
I20260811 05:13:13.449350 10133 heartbeater.cc:347] Connected to a master server at 127.28.143.189:40933
I20260811 05:13:13.449486 10132 heartbeater.cc:347] Connected to a master server at 127.28.143.190:39435
I20260811 05:13:13.449563 10132 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:13.449564 10133 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:13.449770 10132 heartbeater.cc:511] Master 127.28.143.190:39435 requested a full tablet report, sending...
I20260811 05:13:13.449774 10133 heartbeater.cc:511] Master 127.28.143.189:40933 requested a full tablet report, sending...
I20260811 05:13:13.450183 8010 ts_manager.cc:194] Re-registered known tserver with Master: fda127cd3b28435498b97c4445c40438 (127.28.143.130:34837)
I20260811 05:13:13.450268 9194 ts_manager.cc:194] Registered new tserver with Master: fda127cd3b28435498b97c4445c40438 (127.28.143.130:34837)
I20260811 05:13:13.450641 8010 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.143.130@KRBTEST.COM'} at 127.28.143.130:60087
I20260811 05:13:13.470094 10131 heartbeater.cc:347] Connected to a master server at 127.28.143.188:44183
I20260811 05:13:13.470203 10131 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:13.470335 10131 heartbeater.cc:511] Master 127.28.143.188:44183 requested a full tablet report, sending...
I20260811 05:13:13.470789 9526 ts_manager.cc:194] Registered new tserver with Master: fda127cd3b28435498b97c4445c40438 (127.28.143.130:34837)
I20260811 05:13:13.474694 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.131:41421
--local_ip_for_outbound_sockets=127.28.143.131
--tserver_master_addrs=127.28.143.190:39435
--webserver_port=37113
--webserver_interface=127.28.143.131
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.131.keytab
--principal=oryx/127.28.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:43653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.28.143.190:39435,127.28.143.189:40933
--tserver_master_addrs=127.28.143.190:39435,127.28.143.189:40933,127.28.143.188:44183 with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:13.562202 10142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:13.562438 10142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:13.562464 10142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:13.563714 10142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:13.563764 10142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:13.563793 10142 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.131
I20260811 05:13:13.565469 10142 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:43653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.131.keytab
--principal=oryx/127.28.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.131:41421
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.143.131
--webserver_port=37113
--tserver_master_addrs=127.28.143.190:39435,127.28.143.189:40933,127.28.143.188:44183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.131
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:13.565714 10142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:13.565949 10142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:13.571390 10160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:13.571372 10155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:13.571434 10142 server_base.cc:1034] running on GCE node
W20260811 05:13:13.571341 10164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:13.571786 10142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:13.572057 10142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:13.573232 10142 hybrid_clock.cc:648] HybridClock initialized: now 1786425193573215 us; error 37 us; skew 500 ppm
Aug 11 05:13:13 dist-test-slave-f7th krb5kdc[7922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425193, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:13.575987 10142 init.cc:377] Logged in from keytab as oryx/127.28.143.131@KRBTEST.COM (short username oryx)
I20260811 05:13:13.576514 10142 webserver.cc:460] Webserver started at http://127.28.143.131:37113/ using document root <none> and password file <none>
I20260811 05:13:13.576773 10142 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:13.576838 10142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:13.578356 10142 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:13.579056 10173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:13.579227 10142 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:13.579325 10142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-2/wal
uuid: "b94dae50dc884af4b0d3c50374d3872d"
format_stamp: "Formatted at 2026-08-11 05:13:12 on dist-test-slave-f7th"
server_key: "2f47d269b9480b9e2879b41f53ca01ca"
server_key_iv: "6b9a357e44eefc8ce1e114a9f2309ae7"
server_key_version: "encryptionkey@0"
I20260811 05:13:13.579733 10142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:13.582515 9886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.579604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41722 (local address 127.28.143.188:44183)
0811 05:13:13.579647 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:13.579649 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:13.579657 (+ 8us) server_negotiation.cc:407] Connection header received
0811 05:13:13.579684 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:13.579685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:13.579704 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:13.579757 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:13.579911 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.580394 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.581745 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.582176 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.582246 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.582292 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:13.582411 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 05:13:13.582450 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 05:13:13.582543 9136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.579403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:44183 (local address 127.0.0.1:41722)
0811 05:13:13.579446 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:13.579457 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:13.579519 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:13.579773 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:13.579775 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:13.579776 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:13.579832 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.579834 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.580484 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.580486 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.581626 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.581630 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.582268 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.582269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.582338 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.582343 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 05:13:13.582408 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 05:13:13.582462 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260811 05:13:13.582876 9549 raft_consensus.cc:1203] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260811 05:13:13.583074 9603 sys_catalog.cc:455] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc6e53c73bdd4fe18d564e13636cdd5b. Latest consensus state: current_term: 1 leader_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: false } } peers { permanent_uuid: "de49b21914144ffc8824e6f82f4422c4" member_type: NON_VOTER last_known_addr { host: "127.28.143.188" port: 44183 } attrs { promote: true } } }
I20260811 05:13:13.583199 9603 sys_catalog.cc:458] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [sys.catalog]: This master's current role is: LEARNER
I20260811 05:13:13.591785 10142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:13.594645 10142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:13.594822 10142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:13.595129 10142 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:13.595475 10142 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:13.595513 10142 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:13.595543 10142 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:13.595568 10142 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:13.602222 10142 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.131:41421
I20260811 05:13:13.602263 10397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.131:41421 every 8 connection(s)
I20260811 05:13:13.602649 10142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
Aug 11 05:13:13 dist-test-slave-f7th krb5kdc[7922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425193, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.143.131@KRBTEST.COM for oryx/127.28.143.189@KRBTEST.COM
Aug 11 05:13:13 dist-test-slave-f7th krb5kdc[7922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425193, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.143.131@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM
Aug 11 05:13:13 dist-test-slave-f7th krb5kdc[7922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425193, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.143.131@KRBTEST.COM for oryx/127.28.143.188@KRBTEST.COM
I20260811 05:13:13.612064 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 10142
I20260811 05:13:13.615334 9291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.603857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:35529 (local address 127.28.143.189:40933)
0811 05:13:13.603906 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:13.603908 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:13.604030 (+ 122us) server_negotiation.cc:407] Connection header received
0811 05:13:13.605020 (+ 990us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:13.605021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:13.605034 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:13.605098 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:13.606070 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.606581 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.607864 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.608157 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.608212 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.611191 (+ 2979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:13.611193 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:13.611193 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:13.611202 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:13.612967 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.613792 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.613793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.613794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.613860 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.614265 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.614266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.614267 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.614408 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:13.614453 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:13.615138 (+ 685us) server_negotiation.cc:299] Negotiation successful
0811 05:13:13.615231 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 05:13:13.615402 10410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.603952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:44183 (local address 127.28.143.131:46695)
0811 05:13:13.605090 (+ 1138us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:13.605100 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:13.605154 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:13.605332 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:13.605333 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:13.605720 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:13.605993 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.605995 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.606724 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.606725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.607733 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.607736 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.608179 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.608180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.608325 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.609069 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:13.609075 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:13.612050 (+ 2975us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:13.613826 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:13.613827 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:13.613827 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:13.614016 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:13.614216 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:13.614217 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:13.614217 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:13.614272 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:13.614548 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:13.614548 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:13.614733 (+ 185us) client_negotiation.cc:769] Sending connection context
0811 05:13:13.614840 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 05:13:13.615132 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1107,"mutex_wait_us":29,"thread_start_us":56,"threads_started":1}
I20260811 05:13:13.615727 9886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.604034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:46695 (local address 127.28.143.188:44183)
0811 05:13:13.604084 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:13.604085 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:13.605181 (+ 1096us) server_negotiation.cc:407] Connection header received
0811 05:13:13.605211 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:13.605212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:13.605229 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:13.605280 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:13.606134 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.606633 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.607808 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.608096 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.608163 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.612130 (+ 3967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:13.612131 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:13.612132 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:13.612141 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:13.613743 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.614092 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.614093 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.614094 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.614152 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.614333 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.614334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.614334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.614471 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:13.614525 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:13.615545 (+ 1020us) server_negotiation.cc:299] Negotiation successful
0811 05:13:13.615639 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 05:13:13.615816 10406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.603762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.189:40933 (local address 127.28.143.131:35529)
0811 05:13:13.603966 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:13.603975 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:13.604846 (+ 871us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:13.605204 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:13.605209 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:13.605737 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:13.605955 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.605962 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.606693 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.606694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.607736 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.607738 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.608261 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.608262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.608333 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.608992 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:13.609004 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:13.611075 (+ 2071us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:13.613343 (+ 2268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:13.613353 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:13.613355 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:13.613690 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:13.613943 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:13.613944 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:13.613944 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:13.614191 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:13.614502 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:13.614532 (+ 30us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:13.614868 (+ 336us) client_negotiation.cc:769] Sending connection context
0811 05:13:13.615617 (+ 749us) client_negotiation.cc:240] Negotiation successful
0811 05:13:13.615721 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":55,"thread_start_us":76,"threads_started":1}
I20260811 05:13:13.616377 10399 heartbeater.cc:347] Connected to a master server at 127.28.143.188:44183
I20260811 05:13:13.616492 10399 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:13.616720 10399 heartbeater.cc:511] Master 127.28.143.188:44183 requested a full tablet report, sending...
I20260811 05:13:13.616828 8097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.603481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:50093 (local address 127.28.143.190:39435)
0811 05:13:13.603555 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:13.603558 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:13.603783 (+ 225us) server_negotiation.cc:407] Connection header received
0811 05:13:13.605198 (+ 1415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:13.605200 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:13.605213 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:13.605271 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:13.606671 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.607439 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.608248 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.608523 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.608595 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.611636 (+ 3041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:13.611638 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:13.611639 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:13.611649 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:13.614473 (+ 2824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.614839 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.614841 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.614841 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.614917 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.615141 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.615142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.615142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.615291 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:13.615349 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:13.616642 (+ 1293us) server_negotiation.cc:299] Negotiation successful
0811 05:13:13.616732 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 05:13:13.616909 10401 heartbeater.cc:347] Connected to a master server at 127.28.143.189:40933
I20260811 05:13:13.616946 10401 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:13.617005 10401 heartbeater.cc:511] Master 127.28.143.189:40933 requested a full tablet report, sending...
I20260811 05:13:13.617365 9194 ts_manager.cc:194] Registered new tserver with Master: b94dae50dc884af4b0d3c50374d3872d (127.28.143.131:41421)
I20260811 05:13:13.617638 9526 ts_manager.cc:194] Registered new tserver with Master: b94dae50dc884af4b0d3c50374d3872d (127.28.143.131:41421)
I20260811 05:13:13.618304 10405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.603467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.28.143.131:50093)
0811 05:13:13.603648 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:13.603688 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:13.604828 (+ 1140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:13.605285 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:13.605285 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:13.605749 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:13.605955 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.605961 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.607539 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.607541 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.608152 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.608155 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.608597 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.608598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.608708 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.609168 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:13.609172 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:13.611520 (+ 2348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:13.614543 (+ 3023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:13.614544 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:13.614544 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:13.614752 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:13.615004 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:13.615004 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:13.615005 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:13.615073 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:13.615745 (+ 672us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:13.615747 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:13.616332 (+ 585us) client_negotiation.cc:769] Sending connection context
0811 05:13:13.616398 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 05:13:13.617278 (+ 880us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":29,"thread_start_us":94,"threads_started":1}
I20260811 05:13:13.618706 10400 heartbeater.cc:347] Connected to a master server at 127.28.143.190:39435
I20260811 05:13:13.618752 10400 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:13.618815 10400 heartbeater.cc:511] Master 127.28.143.190:39435 requested a full tablet report, sending...
I20260811 05:13:13.619108 8010 ts_manager.cc:194] Re-registered known tserver with Master: b94dae50dc884af4b0d3c50374d3872d (127.28.143.131:41421)
I20260811 05:13:13.619602 8010 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.143.131@KRBTEST.COM'} at 127.28.143.131:50093
I20260811 05:13:13.622368 10424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.613895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42504 (local address 127.28.143.189:40933)
0811 05:13:13.614242 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:13.614244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:13.614251 (+ 7us) server_negotiation.cc:407] Connection header received
0811 05:13:13.614271 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:13.614272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:13.614285 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:13.614345 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:13.615114 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.615909 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.616959 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.617246 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.617302 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.618767 (+ 1465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:13.618769 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:13.618769 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:13.618777 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:13.620749 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.621131 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.621132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.621132 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.621198 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.621388 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.621388 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.621389 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.621549 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:13.621658 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:13.622210 (+ 552us) server_negotiation.cc:299] Negotiation successful
0811 05:13:13.622284 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":51,"threads_started":1}
I20260811 05:13:13.624938 10423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.613648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41736 (local address 127.28.143.188:44183)
0811 05:13:13.614192 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:13.614194 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:13.616227 (+ 2033us) server_negotiation.cc:407] Connection header received
0811 05:13:13.616317 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:13.616318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:13.616331 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:13.616486 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:13.617314 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.618036 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.619527 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.619833 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.619875 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.621898 (+ 2023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:13.621899 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:13.621899 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:13.621908 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:13.623757 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.624147 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.624148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.624149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.624214 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.624414 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.624414 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.624415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.624555 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:13.624697 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:13.624790 (+ 93us) server_negotiation.cc:299] Negotiation successful
0811 05:13:13.624859 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":83,"threads_started":1}
I20260811 05:13:13.626453 10421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.613571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38166 (local address 127.28.143.190:39435)
0811 05:13:13.614604 (+ 1033us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:13.614605 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:13.618410 (+ 3805us) server_negotiation.cc:407] Connection header received
0811 05:13:13.618439 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:13.618440 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:13.618452 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:13.618534 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:13.619515 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.620240 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.621078 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.621355 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.621412 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.623006 (+ 1594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:13.623007 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:13.623008 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:13.623014 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:13.624580 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.625165 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.625166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.625166 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.625289 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.625694 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.625696 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.625696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.625831 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:13.625957 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:13.626251 (+ 294us) server_negotiation.cc:299] Negotiation successful
0811 05:13:13.626360 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1004,"thread_start_us":90,"threads_started":1}
I20260811 05:13:13.628172 8010 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38166:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:13:13.629873 8010 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:13:13.630609 9465 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: false } } peers { permanent_uuid: "de49b21914144ffc8824e6f82f4422c4" member_type: NON_VOTER last_known_addr { host: "127.28.143.188" port: 44183 } attrs { promote: true } } }
I20260811 05:13:13.630720 9465 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:13.631997 10430 raft_consensus.cc:1050] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b: attempting to promote NON_VOTER de49b21914144ffc8824e6f82f4422c4 to VOTER
I20260811 05:13:13.632139 10430 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: false } } peers { permanent_uuid: "de49b21914144ffc8824e6f82f4422c4" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 44183 } attrs { promote: false } }
I20260811 05:13:13.632756 9211 raft_consensus.cc:1261] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [term 1 FOLLOWER]: Refusing update from remote peer dc6e53c73bdd4fe18d564e13636cdd5b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 05:13:13.632822 9549 raft_consensus.cc:1261] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [term 1 LEARNER]: Refusing update from remote peer dc6e53c73bdd4fe18d564e13636cdd5b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 05:13:13.633008 10429 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de49b21914144ffc8824e6f82f4422c4" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 44183 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260811 05:13:13.633133 9465 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260811 05:13:13.633795 9465 raft_consensus.cc:2932] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, de49b21914144ffc8824e6f82f4422c4 (127.28.143.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: false } } peers { permanent_uuid: "de49b21914144ffc8824e6f82f4422c4" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 44183 } attrs { promote: false } } }
I20260811 05:13:13.633908 9211 raft_consensus.cc:2932] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, de49b21914144ffc8824e6f82f4422c4 (127.28.143.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: false } } peers { permanent_uuid: "de49b21914144ffc8824e6f82f4422c4" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 44183 } attrs { promote: false } } }
I20260811 05:13:13.634712 10430 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: false } } peers { permanent_uuid: "de49b21914144ffc8824e6f82f4422c4" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 44183 } attrs { promote: false } } }
I20260811 05:13:13.634807 10430 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:13.634826 9465 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: false } } peers { permanent_uuid: "de49b21914144ffc8824e6f82f4422c4" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 44183 } attrs { promote: false } } }
I20260811 05:13:13.634835 10429 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: false } } peers { permanent_uuid: "de49b21914144ffc8824e6f82f4422c4" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 44183 } attrs { promote: false } } }
I20260811 05:13:13.634853 9549 raft_consensus.cc:2932] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, de49b21914144ffc8824e6f82f4422c4 (127.28.143.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: false } } peers { permanent_uuid: "de49b21914144ffc8824e6f82f4422c4" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 44183 } attrs { promote: false } } }
I20260811 05:13:13.634987 10429 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:13.634917 9465 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:13.636116 9603 sys_catalog.cc:455] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: false } } peers { permanent_uuid: "de49b21914144ffc8824e6f82f4422c4" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 44183 } attrs { promote: false } } }
I20260811 05:13:13.636207 9603 sys_catalog.cc:458] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:13:13.634719 10442 sys_catalog.cc:455] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: false } } peers { permanent_uuid: "de49b21914144ffc8824e6f82f4422c4" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 44183 } attrs { promote: false } } }
I20260811 05:13:13.636855 10442 sys_catalog.cc:458] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:13:13.640125 10451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.636130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60042 (local address 127.28.143.130:34837)
0811 05:13:13.636495 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:13.636500 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:13.636508 (+ 8us) server_negotiation.cc:407] Connection header received
0811 05:13:13.636548 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:13.636550 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:13.636566 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:13.636728 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:13.636869 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.638038 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.638918 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.639774 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.639866 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.639935 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:13.639993 (+ 58us) server_negotiation.cc:299] Negotiation successful
0811 05:13:13.640037 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":77,"threads_started":1}
I20260811 05:13:13.640213 9136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.635742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:39087 (local address 127.0.0.1:55410)
0811 05:13:13.635880 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:13.635892 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:13.635934 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:13.636433 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:13.636435 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:13.636437 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:13.636493 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.636495 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.637951 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.637952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.639094 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.639097 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.639959 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.639961 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.640037 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.640041 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:13:13.640070 (+ 29us) client_negotiation.cc:240] Negotiation successful
0811 05:13:13.640110 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117}
I20260811 05:13:13.640416 10449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.636043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55410 (local address 127.28.143.129:39087)
0811 05:13:13.636233 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:13.636238 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:13.636250 (+ 12us) server_negotiation.cc:407] Connection header received
0811 05:13:13.636294 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:13.636296 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:13.636312 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:13.636396 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:13.636556 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.637859 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.639184 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.639867 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.640192 (+ 325us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.640260 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:13.640290 (+ 30us) server_negotiation.cc:299] Negotiation successful
0811 05:13:13.640333 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":97,"threads_started":1}
I20260811 05:13:13.640465 10447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.635830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.131:41421 (local address 127.0.0.1:41016)
0811 05:13:13.636844 (+ 1014us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:13.636853 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:13.636915 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:13.637229 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:13.637230 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:13.637232 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:13.637277 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.637279 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.638206 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.638207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.639264 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.639266 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.640240 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.640241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.640301 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.640304 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 05:13:13.640363 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 05:13:13.640407 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":998,"thread_start_us":107,"threads_started":1}
I20260811 05:13:13.640125 10448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.635817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:34837 (local address 127.0.0.1:60042)
0811 05:13:13.636326 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:13.636336 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:13.636385 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:13.636735 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:13.636736 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:13.636738 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:13.636780 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.636782 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.638114 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.638116 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.638836 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.638838 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.639840 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.639841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.639908 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.639913 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 05:13:13.639972 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 05:13:13.640022 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":159,"threads_started":1}
I20260811 05:13:13.640793 10452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.636234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41016 (local address 127.28.143.131:41421)
0811 05:13:13.637091 (+ 857us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:13.637095 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:13.637105 (+ 10us) server_negotiation.cc:407] Connection header received
0811 05:13:13.637142 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:13.637144 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:13.637159 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:13.637231 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:13.637389 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.638113 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.639341 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.640064 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.640147 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.640203 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:13.640384 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 05:13:13.640427 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":817,"thread_start_us":78,"threads_started":1}
I20260811 05:13:13.641247 9749 tablet_service.cc:1467] Processing CreateTablet for tablet a85ce8f7174840a386d765caee7b6e51 (DEFAULT_TABLE table=test-table [id=11d801f28deb4d08b6f5bfadc599d0f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:13.641362 10268 tablet_service.cc:1467] Processing CreateTablet for tablet a85ce8f7174840a386d765caee7b6e51 (DEFAULT_TABLE table=test-table [id=11d801f28deb4d08b6f5bfadc599d0f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:13.641635 9749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a85ce8f7174840a386d765caee7b6e51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:13.641685 10268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a85ce8f7174840a386d765caee7b6e51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:13.644599 10454 tablet_bootstrap.cc:492] T a85ce8f7174840a386d765caee7b6e51 P 4c4cc9aaf7b14d99920b5b6edca2a68a: Bootstrap starting.
I20260811 05:13:13.645174 10011 tablet_service.cc:1467] Processing CreateTablet for tablet a85ce8f7174840a386d765caee7b6e51 (DEFAULT_TABLE table=test-table [id=11d801f28deb4d08b6f5bfadc599d0f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:13.645433 10454 tablet_bootstrap.cc:654] T a85ce8f7174840a386d765caee7b6e51 P 4c4cc9aaf7b14d99920b5b6edca2a68a: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:13.645596 10011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a85ce8f7174840a386d765caee7b6e51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:13.645754 10454 log.cc:824] T a85ce8f7174840a386d765caee7b6e51 P 4c4cc9aaf7b14d99920b5b6edca2a68a: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:13.646165 10453 tablet_bootstrap.cc:492] T a85ce8f7174840a386d765caee7b6e51 P b94dae50dc884af4b0d3c50374d3872d: Bootstrap starting.
I20260811 05:13:13.646752 10454 tablet_bootstrap.cc:492] T a85ce8f7174840a386d765caee7b6e51 P 4c4cc9aaf7b14d99920b5b6edca2a68a: No bootstrap required, opened a new log
I20260811 05:13:13.646813 10453 tablet_bootstrap.cc:654] T a85ce8f7174840a386d765caee7b6e51 P b94dae50dc884af4b0d3c50374d3872d: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:13.646821 10454 ts_tablet_manager.cc:1397] T a85ce8f7174840a386d765caee7b6e51 P 4c4cc9aaf7b14d99920b5b6edca2a68a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 05:13:13.647146 10453 log.cc:824] T a85ce8f7174840a386d765caee7b6e51 P b94dae50dc884af4b0d3c50374d3872d: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:13.647971 10453 tablet_bootstrap.cc:492] T a85ce8f7174840a386d765caee7b6e51 P b94dae50dc884af4b0d3c50374d3872d: No bootstrap required, opened a new log
I20260811 05:13:13.648046 10453 ts_tablet_manager.cc:1397] T a85ce8f7174840a386d765caee7b6e51 P b94dae50dc884af4b0d3c50374d3872d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 05:13:13.648510 10454 raft_consensus.cc:348] T a85ce8f7174840a386d765caee7b6e51 P 4c4cc9aaf7b14d99920b5b6edca2a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c4cc9aaf7b14d99920b5b6edca2a68a" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 39087 } } peers { permanent_uuid: "b94dae50dc884af4b0d3c50374d3872d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 41421 } } peers { permanent_uuid: "fda127cd3b28435498b97c4445c40438" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34837 } }
I20260811 05:13:13.648706 10454 raft_consensus.cc:374] T a85ce8f7174840a386d765caee7b6e51 P 4c4cc9aaf7b14d99920b5b6edca2a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:13.648753 10454 raft_consensus.cc:729] T a85ce8f7174840a386d765caee7b6e51 P 4c4cc9aaf7b14d99920b5b6edca2a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c4cc9aaf7b14d99920b5b6edca2a68a, State: Initialized, Role: FOLLOWER
I20260811 05:13:13.648869 10454 consensus_queue.cc:260] T a85ce8f7174840a386d765caee7b6e51 P 4c4cc9aaf7b14d99920b5b6edca2a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c4cc9aaf7b14d99920b5b6edca2a68a" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 39087 } } peers { permanent_uuid: "b94dae50dc884af4b0d3c50374d3872d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 41421 } } peers { permanent_uuid: "fda127cd3b28435498b97c4445c40438" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34837 } }
I20260811 05:13:13.649152 10454 ts_tablet_manager.cc:1428] T a85ce8f7174840a386d765caee7b6e51 P 4c4cc9aaf7b14d99920b5b6edca2a68a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 05:13:13.649271 9882 heartbeater.cc:503] Master 127.28.143.190:39435 was elected leader, sending a full tablet report...
I20260811 05:13:13.650343 10453 raft_consensus.cc:348] T a85ce8f7174840a386d765caee7b6e51 P b94dae50dc884af4b0d3c50374d3872d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c4cc9aaf7b14d99920b5b6edca2a68a" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 39087 } } peers { permanent_uuid: "b94dae50dc884af4b0d3c50374d3872d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 41421 } } peers { permanent_uuid: "fda127cd3b28435498b97c4445c40438" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34837 } }
I20260811 05:13:13.650520 10453 raft_consensus.cc:374] T a85ce8f7174840a386d765caee7b6e51 P b94dae50dc884af4b0d3c50374d3872d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:13.650571 10453 raft_consensus.cc:729] T a85ce8f7174840a386d765caee7b6e51 P b94dae50dc884af4b0d3c50374d3872d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b94dae50dc884af4b0d3c50374d3872d, State: Initialized, Role: FOLLOWER
I20260811 05:13:13.650688 10453 consensus_queue.cc:260] T a85ce8f7174840a386d765caee7b6e51 P b94dae50dc884af4b0d3c50374d3872d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c4cc9aaf7b14d99920b5b6edca2a68a" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 39087 } } peers { permanent_uuid: "b94dae50dc884af4b0d3c50374d3872d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 41421 } } peers { permanent_uuid: "fda127cd3b28435498b97c4445c40438" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34837 } }
I20260811 05:13:13.650949 10453 ts_tablet_manager.cc:1428] T a85ce8f7174840a386d765caee7b6e51 P b94dae50dc884af4b0d3c50374d3872d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 05:13:13.651002 10400 heartbeater.cc:503] Master 127.28.143.190:39435 was elected leader, sending a full tablet report...
I20260811 05:13:13.651176 10462 tablet_bootstrap.cc:492] T a85ce8f7174840a386d765caee7b6e51 P fda127cd3b28435498b97c4445c40438: Bootstrap starting.
I20260811 05:13:13.652007 10462 tablet_bootstrap.cc:654] T a85ce8f7174840a386d765caee7b6e51 P fda127cd3b28435498b97c4445c40438: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:13.652333 10462 log.cc:824] T a85ce8f7174840a386d765caee7b6e51 P fda127cd3b28435498b97c4445c40438: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:13.653193 10462 tablet_bootstrap.cc:492] T a85ce8f7174840a386d765caee7b6e51 P fda127cd3b28435498b97c4445c40438: No bootstrap required, opened a new log
I20260811 05:13:13.653266 10462 ts_tablet_manager.cc:1397] T a85ce8f7174840a386d765caee7b6e51 P fda127cd3b28435498b97c4445c40438: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 05:13:13.655061 10462 raft_consensus.cc:348] T a85ce8f7174840a386d765caee7b6e51 P fda127cd3b28435498b97c4445c40438 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c4cc9aaf7b14d99920b5b6edca2a68a" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 39087 } } peers { permanent_uuid: "b94dae50dc884af4b0d3c50374d3872d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 41421 } } peers { permanent_uuid: "fda127cd3b28435498b97c4445c40438" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34837 } }
I20260811 05:13:13.655201 10462 raft_consensus.cc:374] T a85ce8f7174840a386d765caee7b6e51 P fda127cd3b28435498b97c4445c40438 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:13.655233 10462 raft_consensus.cc:729] T a85ce8f7174840a386d765caee7b6e51 P fda127cd3b28435498b97c4445c40438 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fda127cd3b28435498b97c4445c40438, State: Initialized, Role: FOLLOWER
I20260811 05:13:13.655356 10462 consensus_queue.cc:260] T a85ce8f7174840a386d765caee7b6e51 P fda127cd3b28435498b97c4445c40438 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c4cc9aaf7b14d99920b5b6edca2a68a" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 39087 } } peers { permanent_uuid: "b94dae50dc884af4b0d3c50374d3872d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 41421 } } peers { permanent_uuid: "fda127cd3b28435498b97c4445c40438" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34837 } }
I20260811 05:13:13.655637 10462 ts_tablet_manager.cc:1428] T a85ce8f7174840a386d765caee7b6e51 P fda127cd3b28435498b97c4445c40438: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 05:13:13.655658 10132 heartbeater.cc:503] Master 127.28.143.190:39435 was elected leader, sending a full tablet report...
I20260811 05:13:13.657585 10467 raft_consensus.cc:482] T a85ce8f7174840a386d765caee7b6e51 P fda127cd3b28435498b97c4445c40438 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:13:13.657673 10467 raft_consensus.cc:504] T a85ce8f7174840a386d765caee7b6e51 P fda127cd3b28435498b97c4445c40438 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c4cc9aaf7b14d99920b5b6edca2a68a" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 39087 } } peers { permanent_uuid: "b94dae50dc884af4b0d3c50374d3872d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 41421 } } peers { permanent_uuid: "fda127cd3b28435498b97c4445c40438" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34837 } }
I20260811 05:13:13.657976 10467 leader_election.cc:288] T a85ce8f7174840a386d765caee7b6e51 P fda127cd3b28435498b97c4445c40438 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c4cc9aaf7b14d99920b5b6edca2a68a (127.28.143.129:39087), b94dae50dc884af4b0d3c50374d3872d (127.28.143.131:41421)
I20260811 05:13:13.661975 10138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.658148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:39087 (local address 127.28.143.130:47759)
0811 05:13:13.658218 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:13.658230 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:13.658275 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:13.658402 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:13.658403 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:13.658405 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:13.658473 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.658474 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.659463 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.659464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.660802 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.660806 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.661637 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.661639 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.661733 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.661739 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 05:13:13.661827 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 05:13:13.661881 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":60}
I20260811 05:13:13.661990 10452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.658368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:57973 (local address 127.28.143.131:41421)
0811 05:13:13.658430 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:13.658432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:13.658439 (+ 7us) server_negotiation.cc:407] Connection header received
0811 05:13:13.658461 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:13.658462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:13.658470 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:13.658519 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:13.658670 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.659357 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.660939 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.661510 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.661601 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.661611 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:13.661867 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 05:13:13.661921 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 05:13:13.662027 10449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.658212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:47759 (local address 127.28.143.129:39087)
0811 05:13:13.658262 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:13.658265 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:13.658276 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:13:13.658337 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:13.658337 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:13.658345 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:13.658389 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:13.658572 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.659360 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.660929 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.661533 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.661601 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.661611 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:13.661868 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 05:13:13.661967 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 05:13:13.661998 10140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.658173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.131:41421 (local address 127.28.143.130:57973)
0811 05:13:13.658252 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:13.658260 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:13.658290 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:13.658562 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:13.658563 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:13.658565 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:13.658610 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.658611 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.659469 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.659469 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.660802 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.660806 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.661637 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.661639 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.661733 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.661739 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 05:13:13.661828 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 05:13:13.661882 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67,"mutex_wait_us":39}
I20260811 05:13:13.662832 9787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a85ce8f7174840a386d765caee7b6e51" candidate_uuid: "fda127cd3b28435498b97c4445c40438" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c4cc9aaf7b14d99920b5b6edca2a68a" is_pre_election: true
I20260811 05:13:13.663007 9787 raft_consensus.cc:2454] T a85ce8f7174840a386d765caee7b6e51 P 4c4cc9aaf7b14d99920b5b6edca2a68a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fda127cd3b28435498b97c4445c40438 in term 0.
I20260811 05:13:13.663300 9927 leader_election.cc:302] T a85ce8f7174840a386d765caee7b6e51 P fda127cd3b28435498b97c4445c40438 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c4cc9aaf7b14d99920b5b6edca2a68a, fda127cd3b28435498b97c4445c40438; no voters:
I20260811 05:13:13.663312 10313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a85ce8f7174840a386d765caee7b6e51" candidate_uuid: "fda127cd3b28435498b97c4445c40438" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b94dae50dc884af4b0d3c50374d3872d" is_pre_election: true
I20260811 05:13:13.663444 10313 raft_consensus.cc:2454] T a85ce8f7174840a386d765caee7b6e51 P b94dae50dc884af4b0d3c50374d3872d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fda127cd3b28435498b97c4445c40438 in term 0.
I20260811 05:13:13.663446 10467 raft_consensus.cc:2781] T a85ce8f7174840a386d765caee7b6e51 P fda127cd3b28435498b97c4445c40438 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:13:13.663494 10467 raft_consensus.cc:482] T a85ce8f7174840a386d765caee7b6e51 P fda127cd3b28435498b97c4445c40438 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:13:13.663517 10467 raft_consensus.cc:3037] T a85ce8f7174840a386d765caee7b6e51 P fda127cd3b28435498b97c4445c40438 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:13.664455 10467 raft_consensus.cc:504] T a85ce8f7174840a386d765caee7b6e51 P fda127cd3b28435498b97c4445c40438 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c4cc9aaf7b14d99920b5b6edca2a68a" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 39087 } } peers { permanent_uuid: "b94dae50dc884af4b0d3c50374d3872d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 41421 } } peers { permanent_uuid: "fda127cd3b28435498b97c4445c40438" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34837 } }
I20260811 05:13:13.664657 10467 leader_election.cc:288] T a85ce8f7174840a386d765caee7b6e51 P fda127cd3b28435498b97c4445c40438 [CANDIDATE]: Term 1 election: Requested vote from peers 4c4cc9aaf7b14d99920b5b6edca2a68a (127.28.143.129:39087), b94dae50dc884af4b0d3c50374d3872d (127.28.143.131:41421)
I20260811 05:13:13.664844 9787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a85ce8f7174840a386d765caee7b6e51" candidate_uuid: "fda127cd3b28435498b97c4445c40438" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c4cc9aaf7b14d99920b5b6edca2a68a"
I20260811 05:13:13.664958 9787 raft_consensus.cc:3037] T a85ce8f7174840a386d765caee7b6e51 P 4c4cc9aaf7b14d99920b5b6edca2a68a [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:13.665167 10313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a85ce8f7174840a386d765caee7b6e51" candidate_uuid: "fda127cd3b28435498b97c4445c40438" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b94dae50dc884af4b0d3c50374d3872d"
I20260811 05:13:13.665252 10313 raft_consensus.cc:3037] T a85ce8f7174840a386d765caee7b6e51 P b94dae50dc884af4b0d3c50374d3872d [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:13.665764 9787 raft_consensus.cc:2454] T a85ce8f7174840a386d765caee7b6e51 P 4c4cc9aaf7b14d99920b5b6edca2a68a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fda127cd3b28435498b97c4445c40438 in term 1.
I20260811 05:13:13.665949 9927 leader_election.cc:302] T a85ce8f7174840a386d765caee7b6e51 P fda127cd3b28435498b97c4445c40438 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c4cc9aaf7b14d99920b5b6edca2a68a, fda127cd3b28435498b97c4445c40438; no voters:
I20260811 05:13:13.666059 10467 raft_consensus.cc:2781] T a85ce8f7174840a386d765caee7b6e51 P fda127cd3b28435498b97c4445c40438 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:13.666114 10313 raft_consensus.cc:2454] T a85ce8f7174840a386d765caee7b6e51 P b94dae50dc884af4b0d3c50374d3872d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fda127cd3b28435498b97c4445c40438 in term 1.
I20260811 05:13:13.666237 10467 raft_consensus.cc:686] T a85ce8f7174840a386d765caee7b6e51 P fda127cd3b28435498b97c4445c40438 [term 1 LEADER]: Becoming Leader. State: Replica: fda127cd3b28435498b97c4445c40438, State: Running, Role: LEADER
I20260811 05:13:13.666340 10467 consensus_queue.cc:237] T a85ce8f7174840a386d765caee7b6e51 P fda127cd3b28435498b97c4445c40438 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c4cc9aaf7b14d99920b5b6edca2a68a" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 39087 } } peers { permanent_uuid: "b94dae50dc884af4b0d3c50374d3872d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 41421 } } peers { permanent_uuid: "fda127cd3b28435498b97c4445c40438" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34837 } }
I20260811 05:13:13.667280 8008 catalog_manager.cc:5495] T a85ce8f7174840a386d765caee7b6e51 P fda127cd3b28435498b97c4445c40438 reported cstate change: term changed from 0 to 1, leader changed from <none> to fda127cd3b28435498b97c4445c40438 (127.28.143.130). New cstate: current_term: 1 leader_uuid: "fda127cd3b28435498b97c4445c40438" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c4cc9aaf7b14d99920b5b6edca2a68a" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 39087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b94dae50dc884af4b0d3c50374d3872d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 41421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fda127cd3b28435498b97c4445c40438" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34837 } health_report { overall_health: HEALTHY } } }
I20260811 05:13:13.675529 10451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.672901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60046 (local address 127.28.143.130:34837)
0811 05:13:13.672976 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:13.672978 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:13.672987 (+ 9us) server_negotiation.cc:407] Connection header received
0811 05:13:13.673022 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:13.673023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:13.673041 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:13.673107 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:13:13.673470 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.673953 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.674710 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.674923 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.674985 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.675148 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:13:13.675257 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:13:13.675316 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:13.675430 (+ 114us) server_negotiation.cc:299] Negotiation successful
0811 05:13:13.675449 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 05:13:13.677476 10313 raft_consensus.cc:1261] T a85ce8f7174840a386d765caee7b6e51 P b94dae50dc884af4b0d3c50374d3872d [term 1 FOLLOWER]: Refusing update from remote peer fda127cd3b28435498b97c4445c40438: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:13.677476 9787 raft_consensus.cc:1261] T a85ce8f7174840a386d765caee7b6e51 P 4c4cc9aaf7b14d99920b5b6edca2a68a [term 1 FOLLOWER]: Refusing update from remote peer fda127cd3b28435498b97c4445c40438: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:13.677878 10467 consensus_queue.cc:1035] T a85ce8f7174840a386d765caee7b6e51 P fda127cd3b28435498b97c4445c40438 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b94dae50dc884af4b0d3c50374d3872d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 41421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:13.678035 10468 consensus_queue.cc:1035] T a85ce8f7174840a386d765caee7b6e51 P fda127cd3b28435498b97c4445c40438 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c4cc9aaf7b14d99920b5b6edca2a68a" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:13.681610 10474 mvcc.cc:205] Tried to move back new op lower bound from 7317197593300287488 to 7317197593258156032. Current Snapshot: MvccSnapshot[applied={T|T < 7317197593300287488}]
I20260811 05:13:13.684759 8010 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:13.684861 8010 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:13.686157 8010 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dc6e53c73bdd4fe18d564e13636cdd5b: Sending DeleteTablet for 3 replicas of tablet a85ce8f7174840a386d765caee7b6e51
I20260811 05:13:13.686510 10011 tablet_service.cc:1514] Processing DeleteTablet for tablet a85ce8f7174840a386d765caee7b6e51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:13 UTC) from {username='oryx', principal='oryx/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:60042
I20260811 05:13:13.686573 9749 tablet_service.cc:1514] Processing DeleteTablet for tablet a85ce8f7174840a386d765caee7b6e51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:13 UTC) from {username='oryx', principal='oryx/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:55410
I20260811 05:13:13.686815 10268 tablet_service.cc:1514] Processing DeleteTablet for tablet a85ce8f7174840a386d765caee7b6e51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:13 UTC) from {username='oryx', principal='oryx/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:41016
I20260811 05:13:13.687077 10492 tablet_replica.cc:330] T a85ce8f7174840a386d765caee7b6e51 P fda127cd3b28435498b97c4445c40438: stopping tablet replica
I20260811 05:13:13.687196 10497 tablet_replica.cc:330] T a85ce8f7174840a386d765caee7b6e51 P b94dae50dc884af4b0d3c50374d3872d: stopping tablet replica
I20260811 05:13:13.687453 10497 raft_consensus.cc:2229] T a85ce8f7174840a386d765caee7b6e51 P b94dae50dc884af4b0d3c50374d3872d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:13.687525 10492 raft_consensus.cc:2229] T a85ce8f7174840a386d765caee7b6e51 P fda127cd3b28435498b97c4445c40438 [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:13:13.687582 10497 raft_consensus.cc:2258] T a85ce8f7174840a386d765caee7b6e51 P b94dae50dc884af4b0d3c50374d3872d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:13.687646 10492 raft_consensus.cc:2258] T a85ce8f7174840a386d765caee7b6e51 P fda127cd3b28435498b97c4445c40438 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:13.688189 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 7946
I20260811 05:13:13.688233 10497 ts_tablet_manager.cc:1905] T a85ce8f7174840a386d765caee7b6e51 P b94dae50dc884af4b0d3c50374d3872d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:13.703734 10497 ts_tablet_manager.cc:1918] T a85ce8f7174840a386d765caee7b6e51 P b94dae50dc884af4b0d3c50374d3872d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:13:13.703859 10497 log.cc:1196] T a85ce8f7174840a386d765caee7b6e51 P b94dae50dc884af4b0d3c50374d3872d: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-2/wal/wals/a85ce8f7174840a386d765caee7b6e51
I20260811 05:13:13.704187 10497 ts_tablet_manager.cc:1939] T a85ce8f7174840a386d765caee7b6e51 P b94dae50dc884af4b0d3c50374d3872d: Deleting consensus metadata
I20260811 05:13:13.705011 10494 tablet_replica.cc:330] T a85ce8f7174840a386d765caee7b6e51 P 4c4cc9aaf7b14d99920b5b6edca2a68a: stopping tablet replica
I20260811 05:13:13.705193 10494 raft_consensus.cc:2229] T a85ce8f7174840a386d765caee7b6e51 P 4c4cc9aaf7b14d99920b5b6edca2a68a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:13.705343 10494 raft_consensus.cc:2258] T a85ce8f7174840a386d765caee7b6e51 P 4c4cc9aaf7b14d99920b5b6edca2a68a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:13.708534 10494 ts_tablet_manager.cc:1905] T a85ce8f7174840a386d765caee7b6e51 P 4c4cc9aaf7b14d99920b5b6edca2a68a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:13.710842 10494 ts_tablet_manager.cc:1918] T a85ce8f7174840a386d765caee7b6e51 P 4c4cc9aaf7b14d99920b5b6edca2a68a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:13:13.710947 10494 log.cc:1196] T a85ce8f7174840a386d765caee7b6e51 P 4c4cc9aaf7b14d99920b5b6edca2a68a: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-0/wal/wals/a85ce8f7174840a386d765caee7b6e51
I20260811 05:13:13.711241 10494 ts_tablet_manager.cc:1939] T a85ce8f7174840a386d765caee7b6e51 P 4c4cc9aaf7b14d99920b5b6edca2a68a: Deleting consensus metadata
W20260811 05:13:13.715294 10178 connection.cc:535] server connection from 127.0.0.1:41016 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 05:13:13.715322 9659 connection.cc:535] server connection from 127.0.0.1:55410 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 05:13:13.715776 10423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.687507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41738 (local address 127.28.143.188:44183)
0811 05:13:13.687549 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:13.687551 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:13.704843 (+ 17292us) server_negotiation.cc:407] Connection header received
0811 05:13:13.704903 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:13.704904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:13.704923 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:13.704988 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:13.709063 (+ 4075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.709842 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.711034 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.711337 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.711392 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.713135 (+ 1743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:13.713136 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:13.713136 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:13.713145 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:13.714471 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.714818 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.714819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.714820 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.714877 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.715058 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.715059 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.715059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.715178 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:13.715269 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:13.715484 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 05:13:13.715686 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 05:13:13.715864 10492 ts_tablet_manager.cc:1905] T a85ce8f7174840a386d765caee7b6e51 P fda127cd3b28435498b97c4445c40438: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:13.717909 10492 ts_tablet_manager.cc:1918] T a85ce8f7174840a386d765caee7b6e51 P fda127cd3b28435498b97c4445c40438: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:13:13.718019 10492 log.cc:1196] T a85ce8f7174840a386d765caee7b6e51 P fda127cd3b28435498b97c4445c40438: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-1/wal/wals/a85ce8f7174840a386d765caee7b6e51
I20260811 05:13:13.718315 10492 ts_tablet_manager.cc:1939] T a85ce8f7174840a386d765caee7b6e51 P fda127cd3b28435498b97c4445c40438: Deleting consensus metadata
W20260811 05:13:13.718770 9925 connection.cc:441] server connection from 127.0.0.1:60042 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:60042 (request call id 1) could send its response
I20260811 05:13:13.726334 10423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.719115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41742 (local address 127.28.143.188:44183)
0811 05:13:13.719160 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:13.719162 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:13.719270 (+ 108us) server_negotiation.cc:407] Connection header received
0811 05:13:13.719355 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:13.719356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:13.719367 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:13.719396 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:13.720109 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.720669 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.721712 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.722033 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.722109 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.723657 (+ 1548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:13.723658 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:13.723659 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:13.723668 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:13.725114 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.725517 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.725519 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.725519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.725584 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.725782 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.725783 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.725783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.725926 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:13.726015 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:13.726156 (+ 141us) server_negotiation.cc:299] Negotiation successful
0811 05:13:13.726237 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 05:13:13.727331 10424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.718942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42512 (local address 127.28.143.189:40933)
0811 05:13:13.718999 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:13.719001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:13.719067 (+ 66us) server_negotiation.cc:407] Connection header received
0811 05:13:13.719190 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:13.719192 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:13.719210 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:13.719273 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:13.720081 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.721005 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.722176 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.722505 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.722570 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.724220 (+ 1650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:13.724222 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:13.724222 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:13.724230 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:13.725837 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.726207 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.726208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.726208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.726273 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.726464 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.726465 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.726465 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.726618 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:13.726706 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:13.727133 (+ 427us) server_negotiation.cc:299] Negotiation successful
0811 05:13:13.727228 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 05:13:14.682735 10511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:14.682332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.28.143.130:36263)
0811 05:13:14.682571 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:14.682674 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":117,"threads_started":1}
I20260811 05:13:14.682782 10512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:14.682525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.28.143.131:37891)
0811 05:13:14.682642 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:14.682716 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":56,"threads_started":1}
W20260811 05:13:14.683049 10132 heartbeater.cc:644] Failed to heartbeat to 127.28.143.190:39435 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
W20260811 05:13:14.683131 10400 heartbeater.cc:644] Failed to heartbeat to 127.28.143.190:39435 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
I20260811 05:13:14.683969 10513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:14.683721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.28.143.129:48751)
0811 05:13:14.683833 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:14.683918 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":64,"threads_started":1}
W20260811 05:13:14.684167 9882 heartbeater.cc:644] Failed to heartbeat to 127.28.143.190:39435 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
I20260811 05:13:15.187484 10516 raft_consensus.cc:482] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dc6e53c73bdd4fe18d564e13636cdd5b)
I20260811 05:13:15.187471 10517 raft_consensus.cc:482] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dc6e53c73bdd4fe18d564e13636cdd5b)
I20260811 05:13:15.187655 10516 raft_consensus.cc:504] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: false } } peers { permanent_uuid: "de49b21914144ffc8824e6f82f4422c4" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 44183 } attrs { promote: false } }
I20260811 05:13:15.187697 10517 raft_consensus.cc:504] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: false } } peers { permanent_uuid: "de49b21914144ffc8824e6f82f4422c4" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 44183 } attrs { promote: false } }
I20260811 05:13:15.187978 10517 leader_election.cc:288] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dc6e53c73bdd4fe18d564e13636cdd5b (127.28.143.190:39435), db647a2f54b34f5f8f2bd352ce63e7d9 (127.28.143.189:40933)
I20260811 05:13:15.187985 10516 leader_election.cc:288] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dc6e53c73bdd4fe18d564e13636cdd5b (127.28.143.190:39435), de49b21914144ffc8824e6f82f4422c4 (127.28.143.188:44183)
I20260811 05:13:15.188402 10520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:15.188152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.0.0.1:38644)
0811 05:13:15.188295 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:15.188356 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":60,"threads_started":1}
W20260811 05:13:15.188534 9502 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
I20260811 05:13:15.188649 10521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:15.188153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.0.0.1:38646)
0811 05:13:15.188391 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:15.188445 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":60,"threads_started":1}
W20260811 05:13:15.188891 9170 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
I20260811 05:13:15.189095 10520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:15.188999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.0.0.1:38670)
0811 05:13:15.189041 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:15.189066 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260811 05:13:15.189177 9502 leader_election.cc:334] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer dc6e53c73bdd4fe18d564e13636cdd5b (127.28.143.190:39435): Network error: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
I20260811 05:13:15.189641 10521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:15.189506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.0.0.1:38676)
0811 05:13:15.189572 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:15.189612 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 05:13:15.189755 9170 leader_election.cc:334] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer dc6e53c73bdd4fe18d564e13636cdd5b (127.28.143.190:39435): Network error: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
I20260811 05:13:15.191830 10522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:15.188170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:44183 (local address 127.0.0.1:41752)
0811 05:13:15.188568 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:15.188577 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:15.188710 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:15.188973 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:15.188974 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:15.188977 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:15.189030 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:15.189033 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:15.190066 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:15.190067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:15.190855 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:15.190857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:15.191514 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:15.191516 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:15.191621 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:15.191628 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 05:13:15.191705 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 05:13:15.191756 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":43,"threads_started":1}
I20260811 05:13:15.191857 10527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:15.188368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41752 (local address 127.28.143.188:44183)
0811 05:13:15.188454 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:15.188456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:15.188667 (+ 211us) server_negotiation.cc:407] Connection header received
0811 05:13:15.188845 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:15.188846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:15.188857 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:15.188903 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:15.189108 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:15.189951 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:15.190963 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:15.191395 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:15.191465 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:15.191477 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:15.191727 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 05:13:15.191776 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68,"thread_start_us":41,"threads_started":1}
I20260811 05:13:15.191869 10525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:15.188172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.189:40933 (local address 127.0.0.1:42520)
0811 05:13:15.188503 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:15.188512 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:15.188583 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:15.188889 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:15.188890 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:15.188893 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:15.188958 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:15.188961 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:15.190171 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:15.190173 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:15.191021 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:15.191024 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:15.191636 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:15.191637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:15.191723 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:15.191727 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:13:15.191776 (+ 49us) client_negotiation.cc:240] Negotiation successful
0811 05:13:15.191812 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":57,"threads_started":1}
I20260811 05:13:15.192019 10531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:15.188394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42520 (local address 127.28.143.189:40933)
0811 05:13:15.188523 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:15.188525 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:15.188664 (+ 139us) server_negotiation.cc:407] Connection header received
0811 05:13:15.188702 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:15.188707 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:15.188723 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:15.188795 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:15.189039 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:15.189822 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:15.191121 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:15.191533 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:15.191621 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:15.191631 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:15.191908 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 05:13:15.191959 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":81,"threads_started":1}
I20260811 05:13:15.192512 9211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "de49b21914144ffc8824e6f82f4422c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" is_pre_election: true
I20260811 05:13:15.192526 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "de49b21914144ffc8824e6f82f4422c4" is_pre_election: true
I20260811 05:13:15.192679 9549 raft_consensus.cc:2454] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db647a2f54b34f5f8f2bd352ce63e7d9 in term 1.
I20260811 05:13:15.192679 9211 raft_consensus.cc:2454] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de49b21914144ffc8824e6f82f4422c4 in term 1.
I20260811 05:13:15.192855 9505 leader_election.cc:302] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db647a2f54b34f5f8f2bd352ce63e7d9, de49b21914144ffc8824e6f82f4422c4; no voters: dc6e53c73bdd4fe18d564e13636cdd5b
I20260811 05:13:15.192939 9172 leader_election.cc:302] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db647a2f54b34f5f8f2bd352ce63e7d9, de49b21914144ffc8824e6f82f4422c4; no voters: dc6e53c73bdd4fe18d564e13636cdd5b
I20260811 05:13:15.193058 10516 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 05:13:15.193059 10517 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 05:13:15.193104 10517 raft_consensus.cc:482] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader dc6e53c73bdd4fe18d564e13636cdd5b)
I20260811 05:13:15.193123 10517 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [term 1 FOLLOWER]: Advancing to term 2
I20260811 05:13:15.193105 10516 raft_consensus.cc:482] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader dc6e53c73bdd4fe18d564e13636cdd5b)
I20260811 05:13:15.193296 10516 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [term 1 FOLLOWER]: Advancing to term 2
I20260811 05:13:15.194002 10517 raft_consensus.cc:504] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: false } } peers { permanent_uuid: "de49b21914144ffc8824e6f82f4422c4" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 44183 } attrs { promote: false } }
I20260811 05:13:15.194245 10517 leader_election.cc:288] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [CANDIDATE]: Term 2 election: Requested vote from peers dc6e53c73bdd4fe18d564e13636cdd5b (127.28.143.190:39435), db647a2f54b34f5f8f2bd352ce63e7d9 (127.28.143.189:40933)
I20260811 05:13:15.194312 10516 raft_consensus.cc:504] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: false } } peers { permanent_uuid: "de49b21914144ffc8824e6f82f4422c4" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 44183 } attrs { promote: false } }
I20260811 05:13:15.194460 10516 leader_election.cc:288] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [CANDIDATE]: Term 2 election: Requested vote from peers dc6e53c73bdd4fe18d564e13636cdd5b (127.28.143.190:39435), de49b21914144ffc8824e6f82f4422c4 (127.28.143.188:44183)
I20260811 05:13:15.194585 10525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:15.194275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.0.0.1:38686)
0811 05:13:15.194356 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:15.194392 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 05:13:15.194656 9211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "de49b21914144ffc8824e6f82f4422c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9"
I20260811 05:13:15.194686 9549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "de49b21914144ffc8824e6f82f4422c4"
I20260811 05:13:15.194768 9211 raft_consensus.cc:2379] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate de49b21914144ffc8824e6f82f4422c4 in current term 2: Already voted for candidate db647a2f54b34f5f8f2bd352ce63e7d9 in this term.
I20260811 05:13:15.194780 9549 raft_consensus.cc:2379] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate db647a2f54b34f5f8f2bd352ce63e7d9 in current term 2: Already voted for candidate de49b21914144ffc8824e6f82f4422c4 in this term.
I20260811 05:13:15.194921 10522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:15.194571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.0.0.1:38690)
0811 05:13:15.194724 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:15.194760 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
I20260811 05:13:15.195370 10525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:15.195188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.0.0.1:38702)
0811 05:13:15.195230 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:15.195264 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260811 05:13:15.195454 10522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:15.195302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.0.0.1:38706)
0811 05:13:15.195366 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:15.195434 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 05:13:15.195521 9170 leader_election.cc:334] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer dc6e53c73bdd4fe18d564e13636cdd5b (127.28.143.190:39435): Network error: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
I20260811 05:13:15.195600 9170 leader_election.cc:302] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: db647a2f54b34f5f8f2bd352ce63e7d9; no voters: dc6e53c73bdd4fe18d564e13636cdd5b, de49b21914144ffc8824e6f82f4422c4
W20260811 05:13:15.195621 9502 leader_election.cc:334] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer dc6e53c73bdd4fe18d564e13636cdd5b (127.28.143.190:39435): Network error: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
I20260811 05:13:15.195672 9502 leader_election.cc:302] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de49b21914144ffc8824e6f82f4422c4; no voters: db647a2f54b34f5f8f2bd352ce63e7d9, dc6e53c73bdd4fe18d564e13636cdd5b
I20260811 05:13:15.195674 10516 raft_consensus.cc:2726] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 05:13:15.195739 10517 raft_consensus.cc:2726] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 05:13:15.683831 10534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:15.683510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.28.143.131:40587)
0811 05:13:15.683691 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:15.683730 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":75,"threads_started":1}
I20260811 05:13:15.683754 10535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:15.683510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.28.143.130:33551)
0811 05:13:15.683646 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:15.683697 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":77,"threads_started":1}
I20260811 05:13:15.684886 10536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:15.684682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.28.143.129:38651)
0811 05:13:15.684814 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:15.684850 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":69,"threads_started":1}
I20260811 05:13:16.684684 10537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:16.684399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.28.143.131:48139)
0811 05:13:16.684548 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:16.684590 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":73,"threads_started":1}
I20260811 05:13:16.684738 10540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:16.684507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.28.143.130:47137)
0811 05:13:16.684651 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:16.684688 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":61,"threads_started":1}
I20260811 05:13:16.685514 10541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:16.685305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.28.143.129:54375)
0811 05:13:16.685426 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:16.685465 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":56,"threads_started":1}
I20260811 05:13:16.923879 10542 raft_consensus.cc:482] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:13:16.924023 10542 raft_consensus.cc:504] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: false } } peers { permanent_uuid: "de49b21914144ffc8824e6f82f4422c4" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 44183 } attrs { promote: false } }
I20260811 05:13:16.924252 10542 leader_election.cc:288] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers dc6e53c73bdd4fe18d564e13636cdd5b (127.28.143.190:39435), db647a2f54b34f5f8f2bd352ce63e7d9 (127.28.143.189:40933)
I20260811 05:13:16.924558 9211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "de49b21914144ffc8824e6f82f4422c4" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" is_pre_election: true
I20260811 05:13:16.924667 10543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:16.924412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.0.0.1:38916)
0811 05:13:16.924552 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:16.924590 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":84,"threads_started":1}
I20260811 05:13:16.924683 9211 raft_consensus.cc:2454] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de49b21914144ffc8824e6f82f4422c4 in term 2.
I20260811 05:13:16.924918 9505 leader_election.cc:302] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db647a2f54b34f5f8f2bd352ce63e7d9, de49b21914144ffc8824e6f82f4422c4; no voters:
I20260811 05:13:16.925014 10542 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260811 05:13:16.925060 10542 raft_consensus.cc:482] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:13:16.925079 10542 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [term 2 FOLLOWER]: Advancing to term 3
I20260811 05:13:16.925127 10543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:16.925000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.0.0.1:38928)
0811 05:13:16.925051 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:16.925080 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260811 05:13:16.925200 9502 leader_election.cc:334] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer dc6e53c73bdd4fe18d564e13636cdd5b (127.28.143.190:39435): Network error: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
I20260811 05:13:16.925966 10542 raft_consensus.cc:504] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: false } } peers { permanent_uuid: "de49b21914144ffc8824e6f82f4422c4" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 44183 } attrs { promote: false } }
I20260811 05:13:16.926239 10542 leader_election.cc:288] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [CANDIDATE]: Term 3 election: Requested vote from peers dc6e53c73bdd4fe18d564e13636cdd5b (127.28.143.190:39435), db647a2f54b34f5f8f2bd352ce63e7d9 (127.28.143.189:40933)
I20260811 05:13:16.926515 10543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:16.926373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.0.0.1:38930)
0811 05:13:16.926421 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:16.926490 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260811 05:13:16.926488 9211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "de49b21914144ffc8824e6f82f4422c4" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9"
I20260811 05:13:16.926568 9211 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [term 2 FOLLOWER]: Advancing to term 3
I20260811 05:13:16.926918 10543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:16.926808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.0.0.1:38934)
0811 05:13:16.926867 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:16.926895 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 05:13:16.927006 9502 leader_election.cc:334] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer dc6e53c73bdd4fe18d564e13636cdd5b (127.28.143.190:39435): Network error: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
I20260811 05:13:16.927304 9211 raft_consensus.cc:2454] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de49b21914144ffc8824e6f82f4422c4 in term 3.
I20260811 05:13:16.927527 9505 leader_election.cc:302] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: db647a2f54b34f5f8f2bd352ce63e7d9, de49b21914144ffc8824e6f82f4422c4; no voters: dc6e53c73bdd4fe18d564e13636cdd5b
I20260811 05:13:16.927621 10542 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 05:13:16.927731 10542 raft_consensus.cc:686] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [term 3 LEADER]: Becoming Leader. State: Replica: de49b21914144ffc8824e6f82f4422c4, State: Running, Role: LEADER
I20260811 05:13:16.927798 10542 consensus_queue.cc:237] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: false } } peers { permanent_uuid: "de49b21914144ffc8824e6f82f4422c4" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 44183 } attrs { promote: false } }
I20260811 05:13:16.928176 10546 sys_catalog.cc:455] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de49b21914144ffc8824e6f82f4422c4. Latest consensus state: current_term: 3 leader_uuid: "de49b21914144ffc8824e6f82f4422c4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: false } } peers { permanent_uuid: "de49b21914144ffc8824e6f82f4422c4" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 44183 } attrs { promote: false } } }
I20260811 05:13:16.928295 10546 sys_catalog.cc:458] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:16.928476 10550 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:13:16.929062 10550 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:13:16.929149 10550 catalog_manager.cc:1259] Loaded cluster ID: bbcde4e7dda24824a0a9e147d6d1f4b1
I20260811 05:13:16.929167 10550 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:13:16.929313 10550 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:13:16.929378 10550 catalog_manager.cc:5879] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4: Loaded TSK: 0
I20260811 05:13:16.929479 10550 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:13:16.981738 9526 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41742:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:13:16.983382 9526 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:13:16.984676 10543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:16.984525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.0.0.1:38948)
0811 05:13:16.984566 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:16.984604 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260811 05:13:16.984728 9211 raft_consensus.cc:1261] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [term 3 FOLLOWER]: Refusing update from remote peer de49b21914144ffc8824e6f82f4422c4: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260811 05:13:16.985014 10543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:16.984908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.0.0.1:38954)
0811 05:13:16.984948 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:16.984983 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260811 05:13:16.985005 10546 consensus_queue.cc:1035] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260811 05:13:16.985131 9502 consensus_peers.cc:487] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 -> Peer dc6e53c73bdd4fe18d564e13636cdd5b (127.28.143.190:39435): Couldn't send request to peer dc6e53c73bdd4fe18d564e13636cdd5b. Status: Network error: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 05:13:16.985908 10552 sys_catalog.cc:455] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de49b21914144ffc8824e6f82f4422c4. Latest consensus state: current_term: 3 leader_uuid: "de49b21914144ffc8824e6f82f4422c4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: false } } peers { permanent_uuid: "de49b21914144ffc8824e6f82f4422c4" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 44183 } attrs { promote: false } } }
I20260811 05:13:16.986097 10552 sys_catalog.cc:458] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:13:16.986624 10542 sys_catalog.cc:455] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "de49b21914144ffc8824e6f82f4422c4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: false } } peers { permanent_uuid: "de49b21914144ffc8824e6f82f4422c4" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 44183 } attrs { promote: false } } }
I20260811 05:13:16.986795 10542 sys_catalog.cc:458] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4 [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:16.986922 10552 sys_catalog.cc:455] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "de49b21914144ffc8824e6f82f4422c4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "dc6e53c73bdd4fe18d564e13636cdd5b" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39435 } } peers { permanent_uuid: "db647a2f54b34f5f8f2bd352ce63e7d9" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 40933 } attrs { promote: false } } peers { permanent_uuid: "de49b21914144ffc8824e6f82f4422c4" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 44183 } attrs { promote: false } } }
I20260811 05:13:16.987031 10552 sys_catalog.cc:458] T 00000000000000000000000000000000 P db647a2f54b34f5f8f2bd352ce63e7d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:13:16.993700 10573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:16.989731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55422 (local address 127.28.143.129:39087)
0811 05:13:16.989831 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:16.989833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:16.990013 (+ 180us) server_negotiation.cc:407] Connection header received
0811 05:13:16.990068 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:16.990069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:16.990080 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:16.990290 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:16.990324 (+ 34us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:16.991084 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:16.992678 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:16.993236 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:16.993456 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:16.993476 (+ 20us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:16.993552 (+ 76us) server_negotiation.cc:299] Negotiation successful
0811 05:13:16.993605 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":58,"threads_started":1}
I20260811 05:13:16.993713 10566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:16.989365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:39087 (local address 127.0.0.1:55422)
0811 05:13:16.989958 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:16.989967 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:16.990017 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:16.990173 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:16.990174 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:16.990177 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:16.990230 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:16.990231 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:16.991179 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:16.991180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:16.992298 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:16.992302 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:16.993355 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:16.993356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:16.993474 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:16.993481 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 05:13:16.993543 (+ 62us) client_negotiation.cc:240] Negotiation successful
0811 05:13:16.993576 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"mutex_wait_us":227,"thread_start_us":71,"threads_started":1}
I20260811 05:13:16.994100 10543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:16.989363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:34837 (local address 127.0.0.1:60054)
0811 05:13:16.989493 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:16.989505 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:16.989560 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:16.990243 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:16.990244 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:16.990246 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:16.990301 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:16.990303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:16.991203 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:16.991204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:16.992765 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:16.992769 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:16.993742 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:16.993743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:16.993819 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:16.993825 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 05:13:16.993856 (+ 31us) client_negotiation.cc:240] Negotiation successful
0811 05:13:16.993914 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":1}
I20260811 05:13:16.994125 10567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:16.989485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60054 (local address 127.28.143.130:34837)
0811 05:13:16.990115 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:16.990117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:16.990124 (+ 7us) server_negotiation.cc:407] Connection header received
0811 05:13:16.990148 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:16.990149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:16.990159 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:16.990229 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:16.990385 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:16.991098 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:16.992932 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:16.993630 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:16.993967 (+ 337us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:16.993977 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:16.993994 (+ 17us) server_negotiation.cc:299] Negotiation successful
0811 05:13:16.994042 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":56,"threads_started":1}
I20260811 05:13:16.994344 9749 tablet_service.cc:1467] Processing CreateTablet for tablet efe59a45489a48ce815034acc630ce9e (DEFAULT_TABLE table=test-table [id=8443f6f1ee1a46ab8b165c02c0397f98]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:16.994405 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:16.989495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41026 (local address 127.28.143.131:41421)
0811 05:13:16.989630 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:16.989632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:16.989740 (+ 108us) server_negotiation.cc:407] Connection header received
0811 05:13:16.989777 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:16.989777 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:16.989788 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:16.989908 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:16.990051 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:16.990629 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:16.992419 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:16.993159 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:16.993250 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:16.993262 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:16.994253 (+ 991us) server_negotiation.cc:299] Negotiation successful
0811 05:13:16.994309 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":73,"threads_started":1}
I20260811 05:13:16.994401 10011 tablet_service.cc:1467] Processing CreateTablet for tablet efe59a45489a48ce815034acc630ce9e (DEFAULT_TABLE table=test-table [id=8443f6f1ee1a46ab8b165c02c0397f98]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:16.994614 10011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efe59a45489a48ce815034acc630ce9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:16.994617 9749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efe59a45489a48ce815034acc630ce9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:16.993713 10572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:16.989365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.131:41421 (local address 127.0.0.1:41026)
0811 05:13:16.989676 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:16.989684 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:16.989732 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:16.989912 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:16.989913 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:16.989915 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:16.989985 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:16.989987 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:16.990698 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:16.990699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:16.991626 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:16.991629 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:16.993266 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:16.993267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:16.993347 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:16.993352 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 05:13:16.993513 (+ 161us) client_negotiation.cc:240] Negotiation successful
0811 05:13:16.993572 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"mutex_wait_us":26,"thread_start_us":143,"threads_started":1}
I20260811 05:13:16.996577 10575 tablet_bootstrap.cc:492] T efe59a45489a48ce815034acc630ce9e P 4c4cc9aaf7b14d99920b5b6edca2a68a: Bootstrap starting.
I20260811 05:13:16.996656 10268 tablet_service.cc:1467] Processing CreateTablet for tablet efe59a45489a48ce815034acc630ce9e (DEFAULT_TABLE table=test-table [id=8443f6f1ee1a46ab8b165c02c0397f98]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:16.996706 10574 tablet_bootstrap.cc:492] T efe59a45489a48ce815034acc630ce9e P fda127cd3b28435498b97c4445c40438: Bootstrap starting.
I20260811 05:13:16.996788 10268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efe59a45489a48ce815034acc630ce9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:16.997108 10574 tablet_bootstrap.cc:654] T efe59a45489a48ce815034acc630ce9e P fda127cd3b28435498b97c4445c40438: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:16.997732 10574 tablet_bootstrap.cc:492] T efe59a45489a48ce815034acc630ce9e P fda127cd3b28435498b97c4445c40438: No bootstrap required, opened a new log
I20260811 05:13:16.997786 10574 ts_tablet_manager.cc:1397] T efe59a45489a48ce815034acc630ce9e P fda127cd3b28435498b97c4445c40438: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:16.997913 10574 raft_consensus.cc:348] T efe59a45489a48ce815034acc630ce9e P fda127cd3b28435498b97c4445c40438 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b94dae50dc884af4b0d3c50374d3872d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 41421 } } peers { permanent_uuid: "fda127cd3b28435498b97c4445c40438" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34837 } } peers { permanent_uuid: "4c4cc9aaf7b14d99920b5b6edca2a68a" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 39087 } }
I20260811 05:13:16.997991 10574 raft_consensus.cc:374] T efe59a45489a48ce815034acc630ce9e P fda127cd3b28435498b97c4445c40438 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:16.998045 10574 raft_consensus.cc:729] T efe59a45489a48ce815034acc630ce9e P fda127cd3b28435498b97c4445c40438 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fda127cd3b28435498b97c4445c40438, State: Initialized, Role: FOLLOWER
I20260811 05:13:16.998101 10574 consensus_queue.cc:260] T efe59a45489a48ce815034acc630ce9e P fda127cd3b28435498b97c4445c40438 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b94dae50dc884af4b0d3c50374d3872d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 41421 } } peers { permanent_uuid: "fda127cd3b28435498b97c4445c40438" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34837 } } peers { permanent_uuid: "4c4cc9aaf7b14d99920b5b6edca2a68a" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 39087 } }
I20260811 05:13:16.998292 10574 ts_tablet_manager.cc:1428] T efe59a45489a48ce815034acc630ce9e P fda127cd3b28435498b97c4445c40438: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 05:13:16.998653 10575 tablet_bootstrap.cc:654] T efe59a45489a48ce815034acc630ce9e P 4c4cc9aaf7b14d99920b5b6edca2a68a: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:16.998811 10540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:16.998664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.28.143.130:33129)
0811 05:13:16.998735 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:16.998768 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 05:13:16.998963 10578 tablet_bootstrap.cc:492] T efe59a45489a48ce815034acc630ce9e P b94dae50dc884af4b0d3c50374d3872d: Bootstrap starting.
I20260811 05:13:16.999343 10578 tablet_bootstrap.cc:654] T efe59a45489a48ce815034acc630ce9e P b94dae50dc884af4b0d3c50374d3872d: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:16.999372 10575 tablet_bootstrap.cc:492] T efe59a45489a48ce815034acc630ce9e P 4c4cc9aaf7b14d99920b5b6edca2a68a: No bootstrap required, opened a new log
I20260811 05:13:16.999423 10575 ts_tablet_manager.cc:1397] T efe59a45489a48ce815034acc630ce9e P 4c4cc9aaf7b14d99920b5b6edca2a68a: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260811 05:13:16.999444 10131 heartbeater.cc:503] Master 127.28.143.188:44183 was elected leader, sending a full tablet report...
I20260811 05:13:16.999557 10575 raft_consensus.cc:348] T efe59a45489a48ce815034acc630ce9e P 4c4cc9aaf7b14d99920b5b6edca2a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b94dae50dc884af4b0d3c50374d3872d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 41421 } } peers { permanent_uuid: "fda127cd3b28435498b97c4445c40438" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34837 } } peers { permanent_uuid: "4c4cc9aaf7b14d99920b5b6edca2a68a" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 39087 } }
I20260811 05:13:16.999620 10575 raft_consensus.cc:374] T efe59a45489a48ce815034acc630ce9e P 4c4cc9aaf7b14d99920b5b6edca2a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:16.999639 10575 raft_consensus.cc:729] T efe59a45489a48ce815034acc630ce9e P 4c4cc9aaf7b14d99920b5b6edca2a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c4cc9aaf7b14d99920b5b6edca2a68a, State: Initialized, Role: FOLLOWER
I20260811 05:13:16.999691 10575 consensus_queue.cc:260] T efe59a45489a48ce815034acc630ce9e P 4c4cc9aaf7b14d99920b5b6edca2a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b94dae50dc884af4b0d3c50374d3872d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 41421 } } peers { permanent_uuid: "fda127cd3b28435498b97c4445c40438" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34837 } } peers { permanent_uuid: "4c4cc9aaf7b14d99920b5b6edca2a68a" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 39087 } }
I20260811 05:13:16.999890 10575 ts_tablet_manager.cc:1428] T efe59a45489a48ce815034acc630ce9e P 4c4cc9aaf7b14d99920b5b6edca2a68a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:16.999903 10540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:16.999793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.28.143.130:57173)
0811 05:13:16.999847 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:16.999877 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 05:13:17.000105 10578 tablet_bootstrap.cc:492] T efe59a45489a48ce815034acc630ce9e P b94dae50dc884af4b0d3c50374d3872d: No bootstrap required, opened a new log
I20260811 05:13:17.000162 10578 ts_tablet_manager.cc:1397] T efe59a45489a48ce815034acc630ce9e P b94dae50dc884af4b0d3c50374d3872d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:17.000243 10541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:17.000151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.28.143.129:33207)
0811 05:13:17.000192 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:17.000216 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260811 05:13:17.000280 10578 raft_consensus.cc:348] T efe59a45489a48ce815034acc630ce9e P b94dae50dc884af4b0d3c50374d3872d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b94dae50dc884af4b0d3c50374d3872d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 41421 } } peers { permanent_uuid: "fda127cd3b28435498b97c4445c40438" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34837 } } peers { permanent_uuid: "4c4cc9aaf7b14d99920b5b6edca2a68a" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 39087 } }
I20260811 05:13:17.000348 10578 raft_consensus.cc:374] T efe59a45489a48ce815034acc630ce9e P b94dae50dc884af4b0d3c50374d3872d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:17.000368 10578 raft_consensus.cc:729] T efe59a45489a48ce815034acc630ce9e P b94dae50dc884af4b0d3c50374d3872d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b94dae50dc884af4b0d3c50374d3872d, State: Initialized, Role: FOLLOWER
I20260811 05:13:17.000445 10578 consensus_queue.cc:260] T efe59a45489a48ce815034acc630ce9e P b94dae50dc884af4b0d3c50374d3872d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b94dae50dc884af4b0d3c50374d3872d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 41421 } } peers { permanent_uuid: "fda127cd3b28435498b97c4445c40438" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34837 } } peers { permanent_uuid: "4c4cc9aaf7b14d99920b5b6edca2a68a" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 39087 } }
I20260811 05:13:17.000684 10578 ts_tablet_manager.cc:1428] T efe59a45489a48ce815034acc630ce9e P b94dae50dc884af4b0d3c50374d3872d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 05:13:17.000870 9879 heartbeater.cc:503] Master 127.28.143.188:44183 was elected leader, sending a full tablet report...
I20260811 05:13:17.001063 10537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:17.000909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.28.143.131:34925)
0811 05:13:17.001003 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:17.001036 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 05:13:17.001250 10399 heartbeater.cc:503] Master 127.28.143.188:44183 was elected leader, sending a full tablet report...
I20260811 05:13:17.001428 10537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:17.001259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.28.143.131:35203)
0811 05:13:17.001381 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:17.001408 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
W20260811 05:13:17.018044 9884 tablet.cc:2369] T efe59a45489a48ce815034acc630ce9e P 4c4cc9aaf7b14d99920b5b6edca2a68a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 05:13:17.106189 10403 tablet.cc:2369] T efe59a45489a48ce815034acc630ce9e P b94dae50dc884af4b0d3c50374d3872d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:13:17.175547 10587 raft_consensus.cc:482] T efe59a45489a48ce815034acc630ce9e P b94dae50dc884af4b0d3c50374d3872d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:13:17.175665 10587 raft_consensus.cc:504] T efe59a45489a48ce815034acc630ce9e P b94dae50dc884af4b0d3c50374d3872d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b94dae50dc884af4b0d3c50374d3872d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 41421 } } peers { permanent_uuid: "fda127cd3b28435498b97c4445c40438" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34837 } } peers { permanent_uuid: "4c4cc9aaf7b14d99920b5b6edca2a68a" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 39087 } }
I20260811 05:13:17.176059 10587 leader_election.cc:288] T efe59a45489a48ce815034acc630ce9e P b94dae50dc884af4b0d3c50374d3872d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fda127cd3b28435498b97c4445c40438 (127.28.143.130:34837), 4c4cc9aaf7b14d99920b5b6edca2a68a (127.28.143.129:39087)
I20260811 05:13:17.178964 10573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:17.176350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:33809 (local address 127.28.143.129:39087)
0811 05:13:17.176400 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:17.176403 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:17.176542 (+ 139us) server_negotiation.cc:407] Connection header received
0811 05:13:17.176597 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:17.176598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:17.176611 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:17.176678 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:17.176788 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:17.177246 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:17.178269 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:17.178648 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:17.178732 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:17.178745 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:17.178852 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 05:13:17.178895 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 05:13:17.178993 10588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:17.176280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:39087 (local address 127.28.143.131:33809)
0811 05:13:17.176463 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:17.176473 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:17.176542 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:17.176681 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:17.176682 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:17.176684 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:17.176732 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:17.176733 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:17.177322 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:17.177323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:17.178160 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:17.178164 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:17.178732 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:17.178734 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:17.178790 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:17.178794 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:13:17.178841 (+ 47us) client_negotiation.cc:240] Negotiation successful
0811 05:13:17.178883 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":135,"threads_started":1}
I20260811 05:13:17.179080 10567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:17.176360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:50909 (local address 127.28.143.130:34837)
0811 05:13:17.176420 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:17.176422 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:17.176429 (+ 7us) server_negotiation.cc:407] Connection header received
0811 05:13:17.176455 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:17.176456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:17.176464 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:17.176511 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:17.176685 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:17.177172 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:17.178288 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:17.178719 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:17.178785 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:17.178799 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:17.178964 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 05:13:17.179009 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 05:13:17.179080 10537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:17.176239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:34837 (local address 127.28.143.131:50909)
0811 05:13:17.176303 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:17.176315 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:17.176368 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:17.176526 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:17.176527 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:17.176529 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:17.176604 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:17.176606 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:17.177256 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:17.177257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:17.178172 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:17.178175 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:17.178828 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:17.178829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:17.178887 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:17.178890 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 05:13:17.178934 (+ 44us) client_negotiation.cc:240] Negotiation successful
0811 05:13:17.179006 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 05:13:17.179477 9787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efe59a45489a48ce815034acc630ce9e" candidate_uuid: "b94dae50dc884af4b0d3c50374d3872d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c4cc9aaf7b14d99920b5b6edca2a68a" is_pre_election: true
I20260811 05:13:17.179601 9787 raft_consensus.cc:2454] T efe59a45489a48ce815034acc630ce9e P 4c4cc9aaf7b14d99920b5b6edca2a68a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b94dae50dc884af4b0d3c50374d3872d in term 0.
I20260811 05:13:17.179545 10045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efe59a45489a48ce815034acc630ce9e" candidate_uuid: "b94dae50dc884af4b0d3c50374d3872d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fda127cd3b28435498b97c4445c40438" is_pre_election: true
I20260811 05:13:17.179677 10045 raft_consensus.cc:2454] T efe59a45489a48ce815034acc630ce9e P fda127cd3b28435498b97c4445c40438 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b94dae50dc884af4b0d3c50374d3872d in term 0.
I20260811 05:13:17.179821 10175 leader_election.cc:302] T efe59a45489a48ce815034acc630ce9e P b94dae50dc884af4b0d3c50374d3872d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c4cc9aaf7b14d99920b5b6edca2a68a, b94dae50dc884af4b0d3c50374d3872d; no voters:
I20260811 05:13:17.180007 10587 raft_consensus.cc:2781] T efe59a45489a48ce815034acc630ce9e P b94dae50dc884af4b0d3c50374d3872d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:13:17.180053 10587 raft_consensus.cc:482] T efe59a45489a48ce815034acc630ce9e P b94dae50dc884af4b0d3c50374d3872d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:13:17.180068 10587 raft_consensus.cc:3037] T efe59a45489a48ce815034acc630ce9e P b94dae50dc884af4b0d3c50374d3872d [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:17.180887 10587 raft_consensus.cc:504] T efe59a45489a48ce815034acc630ce9e P b94dae50dc884af4b0d3c50374d3872d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b94dae50dc884af4b0d3c50374d3872d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 41421 } } peers { permanent_uuid: "fda127cd3b28435498b97c4445c40438" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34837 } } peers { permanent_uuid: "4c4cc9aaf7b14d99920b5b6edca2a68a" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 39087 } }
I20260811 05:13:17.181027 10587 leader_election.cc:288] T efe59a45489a48ce815034acc630ce9e P b94dae50dc884af4b0d3c50374d3872d [CANDIDATE]: Term 1 election: Requested vote from peers fda127cd3b28435498b97c4445c40438 (127.28.143.130:34837), 4c4cc9aaf7b14d99920b5b6edca2a68a (127.28.143.129:39087)
I20260811 05:13:17.181232 9787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efe59a45489a48ce815034acc630ce9e" candidate_uuid: "b94dae50dc884af4b0d3c50374d3872d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c4cc9aaf7b14d99920b5b6edca2a68a"
I20260811 05:13:17.181288 10045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efe59a45489a48ce815034acc630ce9e" candidate_uuid: "b94dae50dc884af4b0d3c50374d3872d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fda127cd3b28435498b97c4445c40438"
I20260811 05:13:17.181361 10045 raft_consensus.cc:3037] T efe59a45489a48ce815034acc630ce9e P fda127cd3b28435498b97c4445c40438 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:17.181334 9787 raft_consensus.cc:3037] T efe59a45489a48ce815034acc630ce9e P 4c4cc9aaf7b14d99920b5b6edca2a68a [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:17.182183 10045 raft_consensus.cc:2454] T efe59a45489a48ce815034acc630ce9e P fda127cd3b28435498b97c4445c40438 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b94dae50dc884af4b0d3c50374d3872d in term 1.
I20260811 05:13:17.182183 9787 raft_consensus.cc:2454] T efe59a45489a48ce815034acc630ce9e P 4c4cc9aaf7b14d99920b5b6edca2a68a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b94dae50dc884af4b0d3c50374d3872d in term 1.
I20260811 05:13:17.182403 10175 leader_election.cc:302] T efe59a45489a48ce815034acc630ce9e P b94dae50dc884af4b0d3c50374d3872d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c4cc9aaf7b14d99920b5b6edca2a68a, b94dae50dc884af4b0d3c50374d3872d; no voters:
I20260811 05:13:17.182540 10587 raft_consensus.cc:2781] T efe59a45489a48ce815034acc630ce9e P b94dae50dc884af4b0d3c50374d3872d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:17.182726 10587 raft_consensus.cc:686] T efe59a45489a48ce815034acc630ce9e P b94dae50dc884af4b0d3c50374d3872d [term 1 LEADER]: Becoming Leader. State: Replica: b94dae50dc884af4b0d3c50374d3872d, State: Running, Role: LEADER
I20260811 05:13:17.182804 10587 consensus_queue.cc:237] T efe59a45489a48ce815034acc630ce9e P b94dae50dc884af4b0d3c50374d3872d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b94dae50dc884af4b0d3c50374d3872d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 41421 } } peers { permanent_uuid: "fda127cd3b28435498b97c4445c40438" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34837 } } peers { permanent_uuid: "4c4cc9aaf7b14d99920b5b6edca2a68a" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 39087 } }
I20260811 05:13:17.183146 10537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:17.183047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.28.143.131:40341)
0811 05:13:17.183082 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:17.183119 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260811 05:13:17.183678 9526 catalog_manager.cc:5495] T efe59a45489a48ce815034acc630ce9e P b94dae50dc884af4b0d3c50374d3872d reported cstate change: term changed from 0 to 1, leader changed from <none> to b94dae50dc884af4b0d3c50374d3872d (127.28.143.131). New cstate: current_term: 1 leader_uuid: "b94dae50dc884af4b0d3c50374d3872d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b94dae50dc884af4b0d3c50374d3872d" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 41421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fda127cd3b28435498b97c4445c40438" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c4cc9aaf7b14d99920b5b6edca2a68a" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 39087 } health_report { overall_health: UNKNOWN } } }
W20260811 05:13:17.189916 10136 tablet.cc:2369] T efe59a45489a48ce815034acc630ce9e P fda127cd3b28435498b97c4445c40438: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:13:17.223462 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:17.220849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41034 (local address 127.28.143.131:41421)
0811 05:13:17.220904 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:17.220906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:17.220915 (+ 9us) server_negotiation.cc:407] Connection header received
0811 05:13:17.220962 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:17.220963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:17.220983 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:17.221053 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:13:17.221245 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:17.221732 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:17.222581 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:17.222794 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:17.222861 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:17.223093 (+ 232us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:13:17.223217 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:13:17.223284 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:17.223341 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 05:13:17.223368 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 05:13:17.224767 9787 raft_consensus.cc:1261] T efe59a45489a48ce815034acc630ce9e P 4c4cc9aaf7b14d99920b5b6edca2a68a [term 1 FOLLOWER]: Refusing update from remote peer b94dae50dc884af4b0d3c50374d3872d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:17.224902 10045 raft_consensus.cc:1261] T efe59a45489a48ce815034acc630ce9e P fda127cd3b28435498b97c4445c40438 [term 1 FOLLOWER]: Refusing update from remote peer b94dae50dc884af4b0d3c50374d3872d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:17.225072 10587 consensus_queue.cc:1035] T efe59a45489a48ce815034acc630ce9e P b94dae50dc884af4b0d3c50374d3872d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c4cc9aaf7b14d99920b5b6edca2a68a" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 39087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:17.225234 10591 consensus_queue.cc:1035] T efe59a45489a48ce815034acc630ce9e P b94dae50dc884af4b0d3c50374d3872d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fda127cd3b28435498b97c4445c40438" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:17.226262 10540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:17.226138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.28.143.130:46659)
0811 05:13:17.226188 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:17.226222 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260811 05:13:17.226276 10541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:17.226164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.28.143.129:41131)
0811 05:13:17.226209 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:17.226240 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260811 05:13:17.227118 10597 mvcc.cc:205] Tried to move back new op lower bound from 7317197607830396928 to 7317197607661539328. Current Snapshot: MvccSnapshot[applied={T|T < 7317197607830396928}]
I20260811 05:13:17.227633 10537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:17.227451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.28.143.131:33019)
0811 05:13:17.227561 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:17.227597 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260811 05:13:17.227913 10540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:17.227826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.28.143.130:46575)
0811 05:13:17.227863 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:17.227887 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260811 05:13:17.230134 10537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:17.230026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.28.143.131:45413)
0811 05:13:17.230070 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:17.230098 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260811 05:13:17.230232 10541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:17.230059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39435 (local address 127.28.143.129:60645)
0811 05:13:17.230163 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:17.230194 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:39435: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260811 05:13:17.230468 9526 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41742:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:17.230567 9526 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41742:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:17.232077 9526 catalog_manager.cc:5782] T 00000000000000000000000000000000 P de49b21914144ffc8824e6f82f4422c4: Sending DeleteTablet for 3 replicas of tablet efe59a45489a48ce815034acc630ce9e
I20260811 05:13:17.232403 10268 tablet_service.cc:1514] Processing DeleteTablet for tablet efe59a45489a48ce815034acc630ce9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:17 UTC) from {username='oryx', principal='oryx/127.28.143.188@KRBTEST.COM'} at 127.0.0.1:41026
I20260811 05:13:17.232481 10011 tablet_service.cc:1514] Processing DeleteTablet for tablet efe59a45489a48ce815034acc630ce9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:17 UTC) from {username='oryx', principal='oryx/127.28.143.188@KRBTEST.COM'} at 127.0.0.1:60054
I20260811 05:13:17.232676 9749 tablet_service.cc:1514] Processing DeleteTablet for tablet efe59a45489a48ce815034acc630ce9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:17 UTC) from {username='oryx', principal='oryx/127.28.143.188@KRBTEST.COM'} at 127.0.0.1:55422
I20260811 05:13:17.232885 10607 tablet_replica.cc:330] T efe59a45489a48ce815034acc630ce9e P 4c4cc9aaf7b14d99920b5b6edca2a68a: stopping tablet replica
I20260811 05:13:17.232926 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 9454
I20260811 05:13:17.251317 10606 tablet_replica.cc:330] T efe59a45489a48ce815034acc630ce9e P b94dae50dc884af4b0d3c50374d3872d: stopping tablet replica
I20260811 05:13:17.251580 10606 raft_consensus.cc:2229] T efe59a45489a48ce815034acc630ce9e P b94dae50dc884af4b0d3c50374d3872d [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:13:17.251768 10606 raft_consensus.cc:2258] T efe59a45489a48ce815034acc630ce9e P b94dae50dc884af4b0d3c50374d3872d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:17.254293 10609 tablet_replica.cc:330] T efe59a45489a48ce815034acc630ce9e P fda127cd3b28435498b97c4445c40438: stopping tablet replica
I20260811 05:13:17.254439 10609 raft_consensus.cc:2229] T efe59a45489a48ce815034acc630ce9e P fda127cd3b28435498b97c4445c40438 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:17.254519 10609 raft_consensus.cc:2258] T efe59a45489a48ce815034acc630ce9e P fda127cd3b28435498b97c4445c40438 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:17.257701 10609 ts_tablet_manager.cc:1905] T efe59a45489a48ce815034acc630ce9e P fda127cd3b28435498b97c4445c40438: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:17.259421 10606 ts_tablet_manager.cc:1905] T efe59a45489a48ce815034acc630ce9e P b94dae50dc884af4b0d3c50374d3872d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:17.260931 10609 ts_tablet_manager.cc:1918] T efe59a45489a48ce815034acc630ce9e P fda127cd3b28435498b97c4445c40438: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:13:17.261345 10606 ts_tablet_manager.cc:1918] T efe59a45489a48ce815034acc630ce9e P b94dae50dc884af4b0d3c50374d3872d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:13:17.261438 10606 log.cc:1196] T efe59a45489a48ce815034acc630ce9e P b94dae50dc884af4b0d3c50374d3872d: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-2/wal/wals/efe59a45489a48ce815034acc630ce9e
I20260811 05:13:17.261396 10609 log.cc:1196] T efe59a45489a48ce815034acc630ce9e P fda127cd3b28435498b97c4445c40438: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425159060450-29246-0/minicluster-data/ts-1/wal/wals/efe59a45489a48ce815034acc630ce9e
I20260811 05:13:17.261814 10606 ts_tablet_manager.cc:1939] T efe59a45489a48ce815034acc630ce9e P b94dae50dc884af4b0d3c50374d3872d: Deleting consensus metadata
I20260811 05:13:17.262106 10609 ts_tablet_manager.cc:1939] T efe59a45489a48ce815034acc630ce9e P fda127cd3b28435498b97c4445c40438: Deleting consensus metadata
I20260811 05:13:17.262286 9505 catalog_manager.cc:4841] TS b94dae50dc884af4b0d3c50374d3872d (127.28.143.131:41421): tablet efe59a45489a48ce815034acc630ce9e (table test-table [id=8443f6f1ee1a46ab8b165c02c0397f98]) successfully deleted
I20260811 05:13:17.262787 9502 catalog_manager.cc:4841] TS fda127cd3b28435498b97c4445c40438 (127.28.143.130:34837): tablet efe59a45489a48ce815034acc630ce9e (table test-table [id=8443f6f1ee1a46ab8b165c02c0397f98]) successfully deleted
I20260811 05:13:17.263995 10572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:17.263826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:39087 (local address 127.0.0.1:55430)
0811 05:13:17.263894 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:17.263955 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.129:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 05:13:17.264076 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 9893
W20260811 05:13:17.275130 9504 catalog_manager.cc:4584] TS 4c4cc9aaf7b14d99920b5b6edca2a68a (127.28.143.129:39087): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet efe59a45489a48ce815034acc630ce9e: Network error: Client connection negotiation failed: client connection to 127.28.143.129:39087: connect: Connection refused (error 111)
I20260811 05:13:17.280815 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 10142
I20260811 05:13:17.329618 10572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:17.329417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:39087 (local address 127.0.0.1:55442)
0811 05:13:17.329507 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:17.329568 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.129:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 05:13:17.330025 10572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:17.329936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:39087 (local address 127.0.0.1:55454)
0811 05:13:17.329980 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:17.330005 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.129:39087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260811 05:13:17.333995 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 8937
I20260811 05:13:17.355489 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 9297
2026-08-11T05:13:17Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6916 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:13:17 dist-test-slave-f7th krb5kdc[10615](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:13:17 dist-test-slave-f7th krb5kdc[10615](info): set up 2 sockets
Aug 11 05:13:17 dist-test-slave-f7th krb5kdc[10615](info): commencing operation
krb5kdc: starting...
W20260811 05:13:18.629329 29246 mini_kdc.cc:121] Time spent starting KDC: real 1.232s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Aug 11 05:13:18 dist-test-slave-f7th krb5kdc[10615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:13:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:13:18Z Disabled control of system clock
WARNING: no policy specified for oryx/127.28.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.190.keytab.
Entry for principal oryx/127.28.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.190.keytab.
I20260811 05:13:18.772290 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:39863
--webserver_interface=127.28.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.190.keytab
--principal=oryx/127.28.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:32941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:39863
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:18.863214 10635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:18.863476 10635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:18.863514 10635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:18.864856 10635 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:18.864903 10635 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 05:13:18.864920 10635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:18.864938 10635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:18.864953 10635 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:18.864970 10635 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:13:18.866674 10635 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:32941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:39863
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.190.keytab
--principal=oryx/127.28.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:39863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:18.866982 10635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:18.867214 10635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:18.873317 10642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:18.873304 10651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:18.873304 10644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:18.873644 10635 server_base.cc:1034] running on GCE node
I20260811 05:13:18.873974 10635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:18.874305 10635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:18.875550 10635 hybrid_clock.cc:648] HybridClock initialized: now 1786425198875528 us; error 38 us; skew 500 ppm
Aug 11 05:13:18 dist-test-slave-f7th krb5kdc[10615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425198, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:18.884745 10635 init.cc:377] Logged in from keytab as oryx/127.28.143.190@KRBTEST.COM (short username oryx)
I20260811 05:13:18.885360 10635 webserver.cc:460] Webserver started at http://127.28.143.190:39635/ using document root <none> and password file <none>
I20260811 05:13:18.885594 10635 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:18.885648 10635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:18.885804 10635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:18.886969 10635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "4b9d89233cf04d7697a4dd645a3893ae"
format_stamp: "Formatted at 2026-08-11 05:13:18 on dist-test-slave-f7th"
server_key: "5cd975c912a04bf12dedbd157ad79b2e"
server_key_iv: "b9ec4147bf082e54908fafd7b23d054a"
server_key_version: "encryptionkey@0"
I20260811 05:13:18.887338 10635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "4b9d89233cf04d7697a4dd645a3893ae"
format_stamp: "Formatted at 2026-08-11 05:13:18 on dist-test-slave-f7th"
server_key: "5cd975c912a04bf12dedbd157ad79b2e"
server_key_iv: "b9ec4147bf082e54908fafd7b23d054a"
server_key_version: "encryptionkey@0"
I20260811 05:13:18.888947 10635 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 05:13:18.889942 10666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:18.890146 10635 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:18.890251 10635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "4b9d89233cf04d7697a4dd645a3893ae"
format_stamp: "Formatted at 2026-08-11 05:13:18 on dist-test-slave-f7th"
server_key: "5cd975c912a04bf12dedbd157ad79b2e"
server_key_iv: "b9ec4147bf082e54908fafd7b23d054a"
server_key_version: "encryptionkey@0"
I20260811 05:13:18.890323 10635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:18.906242 10635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:18.909162 10635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:18.909340 10635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:18.915393 10635 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:39863
I20260811 05:13:18.915480 10785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:39863 every 8 connection(s)
I20260811 05:13:18.915879 10635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:13:18.916754 10786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:18.919081 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 10635
I20260811 05:13:18.919183 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:13:18.919414 29246 external_mini_cluster.cc:1428] Setting key 76f35fe3388a61db07c7973f50fdb104
I20260811 05:13:18.919448 10786 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b9d89233cf04d7697a4dd645a3893ae: Bootstrap starting.
I20260811 05:13:18.920322 10786 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b9d89233cf04d7697a4dd645a3893ae: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:18.920691 10786 log.cc:824] T 00000000000000000000000000000000 P 4b9d89233cf04d7697a4dd645a3893ae: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:18.921658 10786 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b9d89233cf04d7697a4dd645a3893ae: No bootstrap required, opened a new log
I20260811 05:13:18.924077 10786 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b9d89233cf04d7697a4dd645a3893ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9d89233cf04d7697a4dd645a3893ae" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39863 } }
I20260811 05:13:18.924240 10786 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b9d89233cf04d7697a4dd645a3893ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:18.924263 10786 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b9d89233cf04d7697a4dd645a3893ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b9d89233cf04d7697a4dd645a3893ae, State: Initialized, Role: FOLLOWER
I20260811 05:13:18.924379 10786 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b9d89233cf04d7697a4dd645a3893ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9d89233cf04d7697a4dd645a3893ae" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39863 } }
I20260811 05:13:18.924454 10786 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b9d89233cf04d7697a4dd645a3893ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:13:18.924489 10786 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b9d89233cf04d7697a4dd645a3893ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:13:18.924536 10786 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b9d89233cf04d7697a4dd645a3893ae [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:18.925352 10786 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b9d89233cf04d7697a4dd645a3893ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9d89233cf04d7697a4dd645a3893ae" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39863 } }
I20260811 05:13:18.925510 10786 leader_election.cc:302] T 00000000000000000000000000000000 P 4b9d89233cf04d7697a4dd645a3893ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b9d89233cf04d7697a4dd645a3893ae; no voters:
I20260811 05:13:18.925715 10786 leader_election.cc:288] T 00000000000000000000000000000000 P 4b9d89233cf04d7697a4dd645a3893ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:13:18.925763 10792 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b9d89233cf04d7697a4dd645a3893ae [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:18.925948 10792 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b9d89233cf04d7697a4dd645a3893ae [term 1 LEADER]: Becoming Leader. State: Replica: 4b9d89233cf04d7697a4dd645a3893ae, State: Running, Role: LEADER
I20260811 05:13:18.926061 10792 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b9d89233cf04d7697a4dd645a3893ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9d89233cf04d7697a4dd645a3893ae" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39863 } }
I20260811 05:13:18.926151 10786 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b9d89233cf04d7697a4dd645a3893ae [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:13:18.926479 10794 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b9d89233cf04d7697a4dd645a3893ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b9d89233cf04d7697a4dd645a3893ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9d89233cf04d7697a4dd645a3893ae" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39863 } } }
I20260811 05:13:18.926581 10794 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b9d89233cf04d7697a4dd645a3893ae [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:18.926487 10795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b9d89233cf04d7697a4dd645a3893ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b9d89233cf04d7697a4dd645a3893ae. Latest consensus state: current_term: 1 leader_uuid: "4b9d89233cf04d7697a4dd645a3893ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b9d89233cf04d7697a4dd645a3893ae" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39863 } } }
I20260811 05:13:18.926743 10795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b9d89233cf04d7697a4dd645a3893ae [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:18.926882 10797 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 11 05:13:18 dist-test-slave-f7th krb5kdc[10615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM
I20260811 05:13:18.927564 10797 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:13:18.929497 10797 catalog_manager.cc:1347] Generated new cluster ID: 97001a16a0fc4796ac94a05f000fab91
I20260811 05:13:18.929559 10797 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:13:18.931519 10790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:18.920082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57540 (local address 127.28.143.190:39863)
0811 05:13:18.920266 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:18.920275 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:18.920304 (+ 29us) server_negotiation.cc:407] Connection header received
0811 05:13:18.920859 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:18.920868 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:18.920938 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:18.921130 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:18.922002 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.922923 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.924011 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.924602 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.924731 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:18.928022 (+ 3291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:18.928027 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:18.928028 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:18.928042 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:18.930018 (+ 1976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:18.930379 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:18.930383 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:18.930383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:18.930484 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:18.930668 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:18.930669 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:18.930670 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:18.930917 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:18.931053 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:18.931196 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 05:13:18.931331 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":86,"threads_started":1}
I20260811 05:13:18.948850 10797 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:13:18.949616 10797 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:13:18.954902 10797 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b9d89233cf04d7697a4dd645a3893ae: Generated new TSK 0
I20260811 05:13:18.955155 10797 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.28.143.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.143.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.143.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.129.keytab.
Entry for principal oryx/127.28.143.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.129.keytab.
I20260811 05:13:18.996726 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.129:0
--local_ip_for_outbound_sockets=127.28.143.129
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:39863
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.129.keytab
--principal=oryx/127.28.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:32941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:19.093010 10836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:19.093263 10836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:19.093290 10836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:19.094630 10836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:19.094687 10836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:19.094708 10836 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 05:13:19.094726 10836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.129
I20260811 05:13:19.096390 10836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:32941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.129.keytab
--principal=oryx/127.28.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.143.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.143.190:39863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.129
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:19.096698 10836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:19.096947 10836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:19.102748 10856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:19.102777 10849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:19.102748 10853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:19.102840 10836 server_base.cc:1034] running on GCE node
I20260811 05:13:19.103219 10836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:19.103566 10836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:19.104722 10836 hybrid_clock.cc:648] HybridClock initialized: now 1786425199104704 us; error 42 us; skew 500 ppm
Aug 11 05:13:19 dist-test-slave-f7th krb5kdc[10615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425199, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:19.107824 10836 init.cc:377] Logged in from keytab as oryx/127.28.143.129@KRBTEST.COM (short username oryx)
I20260811 05:13:19.108474 10836 webserver.cc:460] Webserver started at http://127.28.143.129:34691/ using document root <none> and password file <none>
I20260811 05:13:19.108810 10836 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:19.108892 10836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:19.109051 10836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:19.110984 10836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-0/data/instance:
uuid: "9dbce36fab4e4dd695c4d4d879e301c5"
format_stamp: "Formatted at 2026-08-11 05:13:19 on dist-test-slave-f7th"
server_key: "e3eb14fe2b995ffeade65649eba9a463"
server_key_iv: "7ed869040bd5d9e44f35dfe5ba5f5b54"
server_key_version: "encryptionkey@0"
I20260811 05:13:19.111474 10836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance:
uuid: "9dbce36fab4e4dd695c4d4d879e301c5"
format_stamp: "Formatted at 2026-08-11 05:13:19 on dist-test-slave-f7th"
server_key: "e3eb14fe2b995ffeade65649eba9a463"
server_key_iv: "7ed869040bd5d9e44f35dfe5ba5f5b54"
server_key_version: "encryptionkey@0"
I20260811 05:13:19.113500 10836 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260811 05:13:19.116340 10869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:19.116568 10836 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:13:19.116691 10836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-0/wal
uuid: "9dbce36fab4e4dd695c4d4d879e301c5"
format_stamp: "Formatted at 2026-08-11 05:13:19 on dist-test-slave-f7th"
server_key: "e3eb14fe2b995ffeade65649eba9a463"
server_key_iv: "7ed869040bd5d9e44f35dfe5ba5f5b54"
server_key_version: "encryptionkey@0"
I20260811 05:13:19.116786 10836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:19.145646 10836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:19.148234 10836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:19.148375 10836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:19.149040 10836 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:19.149093 10836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:19.149117 10836 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:19.149129 10836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 05:13:19 dist-test-slave-f7th krb5kdc[10615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425199, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.143.129@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM
I20260811 05:13:19.156984 10836 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.129:37049
I20260811 05:13:19.157078 11096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.129:37049 every 8 connection(s)
I20260811 05:13:19.157526 10836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
I20260811 05:13:19.159994 10790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:19.149865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:38477 (local address 127.28.143.190:39863)
0811 05:13:19.149943 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:19.149945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:19.150018 (+ 73us) server_negotiation.cc:407] Connection header received
0811 05:13:19.150563 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:19.150564 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:19.150578 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:19.150632 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:19.151419 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.152234 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.153488 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.153697 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.153761 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:19.156777 (+ 3016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:19.156780 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:19.156780 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:19.156791 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:19.158593 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:19.159086 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:19.159088 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:19.159088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:19.159154 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:19.159384 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:19.159385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:19.159386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:19.159518 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:19.159571 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:19.159804 (+ 233us) server_negotiation.cc:299] Negotiation successful
0811 05:13:19.159920 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 05:13:19.160133 10884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:19.149625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39863 (local address 127.28.143.129:38477)
0811 05:13:19.149806 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:19.149926 (+ 120us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:19.150446 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:19.150708 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:19.150712 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:19.151012 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:19.151290 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:19.151298 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.152374 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.152375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:19.153378 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:19.153382 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.153786 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.153787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:19.153939 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:19.154506 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:19.154516 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:19.156603 (+ 2087us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:19.158701 (+ 2098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:19.158708 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:19.158711 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:19.158995 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:19.159221 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:19.159221 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:19.159222 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:19.159328 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:19.159568 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:19.159572 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:19.159700 (+ 128us) client_negotiation.cc:769] Sending connection context
0811 05:13:19.159784 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 05:13:19.159924 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":74,"threads_started":1}
I20260811 05:13:19.160703 11101 heartbeater.cc:347] Connected to a master server at 127.28.143.190:39863
I20260811 05:13:19.160807 11101 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:19.160996 11101 heartbeater.cc:511] Master 127.28.143.190:39863 requested a full tablet report, sending...
I20260811 05:13:19.161641 10695 ts_manager.cc:194] Registered new tserver with Master: 9dbce36fab4e4dd695c4d4d879e301c5 (127.28.143.129:37049)
I20260811 05:13:19.162272 10695 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.143.129@KRBTEST.COM'} at 127.28.143.129:38477
I20260811 05:13:19.167536 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 10836
I20260811 05:13:19.167701 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance
I20260811 05:13:19.167933 29246 external_mini_cluster.cc:1428] Setting key c9c13ed401b375d487cc7c63c1838e49
I20260811 05:13:19.170624 11107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:19.162618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39863 (local address 127.28.143.129:35233)
0811 05:13:19.162795 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:19.162806 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:19.162859 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:19.162987 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:19.162987 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:19.163158 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:19.163212 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:19.163214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.163964 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.163967 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:19.164600 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:19.164603 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.165223 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.165225 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:19.165309 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:19.166222 (+ 913us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:19.166229 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:19.166767 (+ 538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:19.169244 (+ 2477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:19.169247 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:19.169247 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:19.169523 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:19.169863 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:19.169864 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:19.169864 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:19.169938 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:19.170213 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:19.170215 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:19.170280 (+ 65us) client_negotiation.cc:769] Sending connection context
0811 05:13:19.170340 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 05:13:19.170497 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":78,"threads_started":1}
I20260811 05:13:19.170625 10790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:19.162680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:35233 (local address 127.28.143.190:39863)
0811 05:13:19.162723 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:19.162725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:19.162870 (+ 145us) server_negotiation.cc:407] Connection header received
0811 05:13:19.162905 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:19.162906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:19.162919 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:19.162984 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:19.163272 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.163744 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.164822 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.165057 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.165194 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:19.166928 (+ 1734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:19.166930 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:19.166931 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:19.166940 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:19.169081 (+ 2141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:19.169697 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:19.169700 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:19.169700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:19.169791 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:19.169997 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:19.169998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:19.169998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:19.170149 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:19.170219 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:19.170375 (+ 156us) server_negotiation.cc:299] Negotiation successful
0811 05:13:19.170499 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
WARNING: no policy specified for oryx/127.28.143.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.143.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.143.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.130.keytab.
Entry for principal oryx/127.28.143.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.130.keytab.
I20260811 05:13:19.200416 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.130:0
--local_ip_for_outbound_sockets=127.28.143.130
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:39863
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.130.keytab
--principal=oryx/127.28.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:32941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:19.287276 11111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:19.287554 11111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:19.287595 11111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:19.288966 11111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:19.289021 11111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:19.289059 11111 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 05:13:19.289086 11111 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.130
I20260811 05:13:19.290871 11111 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:32941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.130.keytab
--principal=oryx/127.28.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.143.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.143.190:39863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.130
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:19.291170 11111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:19.291414 11111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:19.296892 11121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:19.296891 11126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:19.296891 11120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:19.297266 11111 server_base.cc:1034] running on GCE node
I20260811 05:13:19.297482 11111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:19.297740 11111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:19.298934 11111 hybrid_clock.cc:648] HybridClock initialized: now 1786425199298884 us; error 33 us; skew 500 ppm
Aug 11 05:13:19 dist-test-slave-f7th krb5kdc[10615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425199, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:19.301671 11111 init.cc:377] Logged in from keytab as oryx/127.28.143.130@KRBTEST.COM (short username oryx)
I20260811 05:13:19.302189 11111 webserver.cc:460] Webserver started at http://127.28.143.130:46295/ using document root <none> and password file <none>
I20260811 05:13:19.302428 11111 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:19.302477 11111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:19.302613 11111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:19.303714 11111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-1/data/instance:
uuid: "8f7748967c7344fca44f6bfe49a720de"
format_stamp: "Formatted at 2026-08-11 05:13:19 on dist-test-slave-f7th"
server_key: "7967ce724b99bf6c90c849f19e3aa73d"
server_key_iv: "7090efbd147226991bf450f6e2d3c49b"
server_key_version: "encryptionkey@0"
I20260811 05:13:19.304070 11111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance:
uuid: "8f7748967c7344fca44f6bfe49a720de"
format_stamp: "Formatted at 2026-08-11 05:13:19 on dist-test-slave-f7th"
server_key: "7967ce724b99bf6c90c849f19e3aa73d"
server_key_iv: "7090efbd147226991bf450f6e2d3c49b"
server_key_version: "encryptionkey@0"
I20260811 05:13:19.305563 11111 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 05:13:19.306393 11139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:19.306564 11111 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:19.306661 11111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-1/wal
uuid: "8f7748967c7344fca44f6bfe49a720de"
format_stamp: "Formatted at 2026-08-11 05:13:19 on dist-test-slave-f7th"
server_key: "7967ce724b99bf6c90c849f19e3aa73d"
server_key_iv: "7090efbd147226991bf450f6e2d3c49b"
server_key_version: "encryptionkey@0"
I20260811 05:13:19.306725 11111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:19.328702 11111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:19.331099 11111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:19.331254 11111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:19.331791 11111 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:19.331838 11111 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:19.331872 11111 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:19.331889 11111 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 05:13:19 dist-test-slave-f7th krb5kdc[10615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425199, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.143.130@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM
I20260811 05:13:19.339702 11111 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.130:41671
I20260811 05:13:19.339783 11361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.130:41671 every 8 connection(s)
I20260811 05:13:19.340126 11111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
I20260811 05:13:19.342585 10790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:19.332304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:41961 (local address 127.28.143.190:39863)
0811 05:13:19.332379 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:19.332382 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:19.332593 (+ 211us) server_negotiation.cc:407] Connection header received
0811 05:13:19.333146 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:19.333148 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:19.333162 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:19.333234 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:19.333948 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.335019 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.336110 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.336326 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.336390 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:19.339606 (+ 3216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:19.339608 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:19.339608 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:19.339618 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:19.341215 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:19.341630 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:19.341631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:19.341632 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:19.341692 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:19.341918 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:19.341919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:19.341920 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:19.342057 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:19.342116 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:19.342391 (+ 275us) server_negotiation.cc:299] Negotiation successful
0811 05:13:19.342478 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 05:13:19.342805 11154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:19.332260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39863 (local address 127.28.143.130:41961)
0811 05:13:19.332446 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:19.332499 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:19.333039 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:19.333314 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:19.333318 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:19.333648 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:19.333861 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:19.333866 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.335142 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.335143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:19.336011 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:19.336015 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.336410 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.336411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:19.336547 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:19.337268 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:19.337280 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:19.339473 (+ 2193us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:19.341304 (+ 1831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:19.341311 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:19.341314 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:19.341557 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:19.341748 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:19.341748 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:19.341749 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:19.341861 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:19.342114 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:19.342122 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:19.342267 (+ 145us) client_negotiation.cc:769] Sending connection context
0811 05:13:19.342361 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 05:13:19.342578 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":99,"threads_started":1}
I20260811 05:13:19.343276 11362 heartbeater.cc:347] Connected to a master server at 127.28.143.190:39863
I20260811 05:13:19.343374 11362 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:19.343590 11362 heartbeater.cc:511] Master 127.28.143.190:39863 requested a full tablet report, sending...
I20260811 05:13:19.344041 10694 ts_manager.cc:194] Registered new tserver with Master: 8f7748967c7344fca44f6bfe49a720de (127.28.143.130:41671)
I20260811 05:13:19.344533 10694 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.143.130@KRBTEST.COM'} at 127.28.143.130:41961
I20260811 05:13:19.346997 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 11111
I20260811 05:13:19.347118 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance
I20260811 05:13:19.347317 29246 external_mini_cluster.cc:1428] Setting key 534de45861b39546bae263dbb4108d17
I20260811 05:13:19.350431 10790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:19.344435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:41343 (local address 127.28.143.190:39863)
0811 05:13:19.344481 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:19.344483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:19.344643 (+ 160us) server_negotiation.cc:407] Connection header received
0811 05:13:19.344694 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:19.344695 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:19.344706 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:19.344764 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:19.345133 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.345606 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.346285 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.346536 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.346607 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:19.347745 (+ 1138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:19.347746 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:19.347746 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:19.347752 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:19.349296 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:19.349659 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:19.349660 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:19.349660 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:19.349718 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:19.349883 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:19.349884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:19.349884 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:19.350045 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:19.350106 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:19.350250 (+ 144us) server_negotiation.cc:299] Negotiation successful
0811 05:13:19.350338 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 05:13:19.350456 11375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:19.344365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39863 (local address 127.28.143.130:41343)
0811 05:13:19.344545 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:19.344555 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:19.344643 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:19.344768 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:19.344769 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:19.344994 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:19.345061 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:19.345064 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.345693 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.345694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:19.346187 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:19.346188 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.346631 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.346631 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:19.346687 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:19.347250 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:19.347255 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:19.347668 (+ 413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:19.349382 (+ 1714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:19.349383 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:19.349384 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:19.349590 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:19.349773 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:19.349773 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:19.349774 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:19.349834 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:19.350108 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:19.350109 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:19.350161 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 05:13:19.350220 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 05:13:19.350333 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":125,"threads_started":1}
WARNING: no policy specified for oryx/127.28.143.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.143.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.143.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.131.keytab.
Entry for principal oryx/127.28.143.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.131.keytab.
I20260811 05:13:19.381036 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.131:0
--local_ip_for_outbound_sockets=127.28.143.131
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:39863
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.131.keytab
--principal=oryx/127.28.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:32941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:19.469117 11380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:19.469412 11380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:19.469444 11380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:19.470769 11380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:19.470825 11380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:19.470847 11380 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 05:13:19.470865 11380 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.131
I20260811 05:13:19.472738 11380 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:32941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.131.keytab
--principal=oryx/127.28.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.143.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.143.190:39863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.131
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:19.473014 11380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:19.473270 11380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:19.480414 11392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:19.480487 11380 server_base.cc:1034] running on GCE node
W20260811 05:13:19.480564 11393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:19.480427 11403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:19.480940 11380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:19.481227 11380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:19.482434 11380 hybrid_clock.cc:648] HybridClock initialized: now 1786425199482410 us; error 39 us; skew 500 ppm
Aug 11 05:13:19 dist-test-slave-f7th krb5kdc[10615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425199, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:19.485452 11380 init.cc:377] Logged in from keytab as oryx/127.28.143.131@KRBTEST.COM (short username oryx)
I20260811 05:13:19.485954 11380 webserver.cc:460] Webserver started at http://127.28.143.131:40485/ using document root <none> and password file <none>
I20260811 05:13:19.486176 11380 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:19.486228 11380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:19.486366 11380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:19.487517 11380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-2/data/instance:
uuid: "a0c7e62d8f73465585819096c63c4a53"
format_stamp: "Formatted at 2026-08-11 05:13:19 on dist-test-slave-f7th"
server_key: "f44900ae4198ead34531e26d7fa74048"
server_key_iv: "1dd10349939c3fc7bfe7d66d0fe02ea6"
server_key_version: "encryptionkey@0"
I20260811 05:13:19.487862 11380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance:
uuid: "a0c7e62d8f73465585819096c63c4a53"
format_stamp: "Formatted at 2026-08-11 05:13:19 on dist-test-slave-f7th"
server_key: "f44900ae4198ead34531e26d7fa74048"
server_key_iv: "1dd10349939c3fc7bfe7d66d0fe02ea6"
server_key_version: "encryptionkey@0"
I20260811 05:13:19.489326 11380 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 05:13:19.490163 11418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:19.490401 11380 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:19.490518 11380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-2/wal
uuid: "a0c7e62d8f73465585819096c63c4a53"
format_stamp: "Formatted at 2026-08-11 05:13:19 on dist-test-slave-f7th"
server_key: "f44900ae4198ead34531e26d7fa74048"
server_key_iv: "1dd10349939c3fc7bfe7d66d0fe02ea6"
server_key_version: "encryptionkey@0"
I20260811 05:13:19.490597 11380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:19.500789 11380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:19.502879 11380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:19.503039 11380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:19.503574 11380 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:19.503613 11380 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:19.503644 11380 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:19.503661 11380 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:19.510849 11380 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.131:40121
I20260811 05:13:19.510946 11646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.131:40121 every 8 connection(s)
Aug 11 05:13:19 dist-test-slave-f7th krb5kdc[10615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425199, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.143.131@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM
I20260811 05:13:19.511283 11380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
I20260811 05:13:19.515033 10790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:19.504254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:44621 (local address 127.28.143.190:39863)
0811 05:13:19.504307 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:19.504310 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:19.504494 (+ 184us) server_negotiation.cc:407] Connection header received
0811 05:13:19.505110 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:19.505111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:19.505124 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:19.505207 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:19.506143 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.506998 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.508489 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.508766 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.508825 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:19.511592 (+ 2767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:19.511594 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:19.511594 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:19.511611 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:19.513198 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:19.513777 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:19.513778 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:19.513779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:19.513842 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:19.514214 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:19.514216 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:19.514216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:19.514466 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:19.514537 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:19.514875 (+ 338us) server_negotiation.cc:299] Negotiation successful
0811 05:13:19.514959 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 05:13:19.515216 11435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:19.504224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39863 (local address 127.28.143.131:44621)
0811 05:13:19.504379 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:19.504416 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:19.504979 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:19.505294 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:19.505300 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:19.505680 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:19.505990 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:19.505998 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.507091 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.507093 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:19.508389 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:19.508393 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.508880 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.508881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:19.509078 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:19.509810 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:19.509819 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:19.511483 (+ 1664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:19.513324 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:19.513332 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:19.513334 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:19.513673 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:19.513929 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:19.513930 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:19.513930 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:19.514105 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:19.514564 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:19.514571 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:19.514712 (+ 141us) client_negotiation.cc:769] Sending connection context
0811 05:13:19.514812 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 05:13:19.514991 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":80,"threads_started":1}
I20260811 05:13:19.515852 11647 heartbeater.cc:347] Connected to a master server at 127.28.143.190:39863
I20260811 05:13:19.515955 11647 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:19.516172 11647 heartbeater.cc:511] Master 127.28.143.190:39863 requested a full tablet report, sending...
I20260811 05:13:19.516656 10694 ts_manager.cc:194] Registered new tserver with Master: a0c7e62d8f73465585819096c63c4a53 (127.28.143.131:40121)
I20260811 05:13:19.517170 10694 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.143.131@KRBTEST.COM'} at 127.28.143.131:44621
I20260811 05:13:19.517494 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 11380
I20260811 05:13:19.517589 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance
I20260811 05:13:19.517781 29246 external_mini_cluster.cc:1428] Setting key de632a846bb2c0f96f1bc847558d6a62
I20260811 05:13:19.518599 29246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:13:19.523316 10790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:19.517184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:39031 (local address 127.28.143.190:39863)
0811 05:13:19.517225 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:19.517226 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:19.517291 (+ 65us) server_negotiation.cc:407] Connection header received
0811 05:13:19.517320 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:19.517320 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:19.517331 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:19.517377 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:19.517791 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.518442 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.519211 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.519437 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.519535 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:19.520748 (+ 1213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:19.520749 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:19.520749 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:19.520757 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:19.522104 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:19.522510 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:19.522511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:19.522511 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:19.522606 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:19.522814 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:19.522814 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:19.522814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:19.522975 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:19.523028 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:19.523182 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 05:13:19.523247 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 05:13:19.523378 11657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:19.517068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39863 (local address 127.28.143.131:39031)
0811 05:13:19.517200 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:19.517212 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:19.517279 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:19.517425 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:19.517427 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:19.517640 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:19.517709 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:19.517711 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.518540 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.518541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:19.519131 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:19.519134 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.519593 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.519594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:19.519677 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:19.520185 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:19.520190 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:19.520666 (+ 476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:19.522209 (+ 1543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:19.522210 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:19.522211 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:19.522412 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:19.522681 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:19.522682 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:19.522682 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:19.522750 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:19.523036 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:19.523037 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:19.523092 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 05:13:19.523165 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 05:13:19.523275 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":71,"threads_started":1}
Aug 11 05:13:19 dist-test-slave-f7th krb5kdc[10615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425199, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 05:13:19 dist-test-slave-f7th krb5kdc[10615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:19 dist-test-slave-f7th krb5kdc[10615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 05:13:19.543764 10790 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:19.538130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57550 (local address 127.28.143.190:39863)
0811 05:13:19.538190 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:19.538192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:19.538201 (+ 9us) server_negotiation.cc:407] Connection header received
0811 05:13:19.538229 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:19.538230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:19.538239 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:19.538297 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:19.538957 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.539470 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.540278 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.540472 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.540532 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:19.543676 (+ 3144us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57550: BlockingRecv error: recv got EOF from 127.0.0.1:57550 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 11 05:13:19 dist-test-slave-f7th krb5kdc[10615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425199, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM
I20260811 05:13:19.552373 10790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:19.544774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57556 (local address 127.28.143.190:39863)
0811 05:13:19.544828 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:19.544830 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:19.544961 (+ 131us) server_negotiation.cc:407] Connection header received
0811 05:13:19.544991 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:19.544992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:19.545002 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:19.545097 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:19.545576 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.546099 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.547008 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.547206 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.547269 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:19.549740 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:19.549741 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:19.549742 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:19.549751 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:19.551149 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:19.551470 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:19.551471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:19.551471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:19.551520 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:19.551790 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:19.551792 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:19.551792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:19.551932 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:19.552026 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:19.552179 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 05:13:19.552251 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 05:13:19.554528 10694 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57556:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:13:19.556273 10694 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:13:19.563761 11692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:19.559004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:41671 (local address 127.0.0.1:42342)
0811 05:13:19.559156 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:19.559201 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:19.559279 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:19.559674 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:19.559678 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:19.559680 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:19.559765 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:19.559767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.561091 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.561092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:19.562419 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:19.562422 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.563396 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.563397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:19.563510 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:19.563529 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 05:13:19.563608 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 05:13:19.563669 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":76,"threads_started":1}
I20260811 05:13:19.563793 11693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:19.559129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42342 (local address 127.28.143.130:41671)
0811 05:13:19.559288 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:19.559293 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:19.559306 (+ 13us) server_negotiation.cc:407] Connection header received
0811 05:13:19.559477 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:19.559482 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:19.559503 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:19.559581 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:19.559837 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.561002 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.562516 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.563307 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.563408 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:19.563473 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:19.563637 (+ 164us) server_negotiation.cc:299] Negotiation successful
0811 05:13:19.563683 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":81,"threads_started":1}
I20260811 05:13:19.564442 11695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:19.560440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.131:40121 (local address 127.0.0.1:36154)
0811 05:13:19.560578 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:19.560588 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:19.560676 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:19.560877 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:19.560878 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:19.560881 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:19.560953 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:19.560955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.562285 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.562286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:19.563117 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:19.563119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.564051 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.564052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:19.564199 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:19.564204 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 05:13:19.564249 (+ 45us) client_negotiation.cc:240] Negotiation successful
0811 05:13:19.564368 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":79,"threads_started":1}
I20260811 05:13:19.564815 11694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:19.559476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36154 (local address 127.28.143.131:40121)
0811 05:13:19.559658 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:19.559662 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:19.560663 (+ 1001us) server_negotiation.cc:407] Connection header received
0811 05:13:19.560748 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:19.560751 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:19.560768 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:19.561031 (+ 263us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:19.561100 (+ 69us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.562201 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.563225 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.563984 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.564059 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:19.564115 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:19.564671 (+ 556us) server_negotiation.cc:299] Negotiation successful
0811 05:13:19.564720 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":100,"threads_started":1}
I20260811 05:13:19.564857 11699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:19.560671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:37049 (local address 127.0.0.1:48010)
0811 05:13:19.560781 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:19.560788 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:19.560827 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:19.561251 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:19.561252 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:19.561254 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:19.561351 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:19.561353 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.562550 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.562550 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:19.563619 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:19.563623 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.564577 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.564578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:19.564708 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:19.564713 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 05:13:19.564745 (+ 32us) client_negotiation.cc:240] Negotiation successful
0811 05:13:19.564788 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":45,"thread_start_us":78,"threads_started":1}
I20260811 05:13:19.564952 11246 tablet_service.cc:1467] Processing CreateTablet for tablet 8bebd7fb7deb4e2caa129de3d356205a (DEFAULT_TABLE table=test-table [id=8107bd8315d64bf39610c87ef8ea5bd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:19.565310 11701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:19.560940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48010 (local address 127.28.143.129:37049)
0811 05:13:19.561099 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:19.561103 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:19.561113 (+ 10us) server_negotiation.cc:407] Connection header received
0811 05:13:19.561150 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:19.561152 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:19.561164 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:19.561233 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:19.561413 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.562460 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.563715 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.564473 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.565059 (+ 586us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:19.565133 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:19.565161 (+ 28us) server_negotiation.cc:299] Negotiation successful
0811 05:13:19.565204 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":83,"threads_started":1}
I20260811 05:13:19.565410 11246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bebd7fb7deb4e2caa129de3d356205a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:19.565423 11518 tablet_service.cc:1467] Processing CreateTablet for tablet 8bebd7fb7deb4e2caa129de3d356205a (DEFAULT_TABLE table=test-table [id=8107bd8315d64bf39610c87ef8ea5bd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:19.565814 11518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bebd7fb7deb4e2caa129de3d356205a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:19.566020 10967 tablet_service.cc:1467] Processing CreateTablet for tablet 8bebd7fb7deb4e2caa129de3d356205a (DEFAULT_TABLE table=test-table [id=8107bd8315d64bf39610c87ef8ea5bd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:19.566378 10967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bebd7fb7deb4e2caa129de3d356205a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:19.569373 11702 tablet_bootstrap.cc:492] T 8bebd7fb7deb4e2caa129de3d356205a P a0c7e62d8f73465585819096c63c4a53: Bootstrap starting.
I20260811 05:13:19.570168 11702 tablet_bootstrap.cc:654] T 8bebd7fb7deb4e2caa129de3d356205a P a0c7e62d8f73465585819096c63c4a53: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:19.570516 11702 log.cc:824] T 8bebd7fb7deb4e2caa129de3d356205a P a0c7e62d8f73465585819096c63c4a53: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:19.570780 11703 tablet_bootstrap.cc:492] T 8bebd7fb7deb4e2caa129de3d356205a P 8f7748967c7344fca44f6bfe49a720de: Bootstrap starting.
I20260811 05:13:19.571295 11706 tablet_bootstrap.cc:492] T 8bebd7fb7deb4e2caa129de3d356205a P 9dbce36fab4e4dd695c4d4d879e301c5: Bootstrap starting.
I20260811 05:13:19.571602 11702 tablet_bootstrap.cc:492] T 8bebd7fb7deb4e2caa129de3d356205a P a0c7e62d8f73465585819096c63c4a53: No bootstrap required, opened a new log
I20260811 05:13:19.571691 11702 ts_tablet_manager.cc:1397] T 8bebd7fb7deb4e2caa129de3d356205a P a0c7e62d8f73465585819096c63c4a53: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 05:13:19.571766 11703 tablet_bootstrap.cc:654] T 8bebd7fb7deb4e2caa129de3d356205a P 8f7748967c7344fca44f6bfe49a720de: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:19.571985 11706 tablet_bootstrap.cc:654] T 8bebd7fb7deb4e2caa129de3d356205a P 9dbce36fab4e4dd695c4d4d879e301c5: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:19.572081 11703 log.cc:824] T 8bebd7fb7deb4e2caa129de3d356205a P 8f7748967c7344fca44f6bfe49a720de: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:19.572300 11706 log.cc:824] T 8bebd7fb7deb4e2caa129de3d356205a P 9dbce36fab4e4dd695c4d4d879e301c5: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:19.572922 11703 tablet_bootstrap.cc:492] T 8bebd7fb7deb4e2caa129de3d356205a P 8f7748967c7344fca44f6bfe49a720de: No bootstrap required, opened a new log
I20260811 05:13:19.573001 11703 ts_tablet_manager.cc:1397] T 8bebd7fb7deb4e2caa129de3d356205a P 8f7748967c7344fca44f6bfe49a720de: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 05:13:19.573371 11706 tablet_bootstrap.cc:492] T 8bebd7fb7deb4e2caa129de3d356205a P 9dbce36fab4e4dd695c4d4d879e301c5: No bootstrap required, opened a new log
I20260811 05:13:19.573586 11706 ts_tablet_manager.cc:1397] T 8bebd7fb7deb4e2caa129de3d356205a P 9dbce36fab4e4dd695c4d4d879e301c5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 05:13:19.573951 11702 raft_consensus.cc:348] T 8bebd7fb7deb4e2caa129de3d356205a P a0c7e62d8f73465585819096c63c4a53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } } peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } }
I20260811 05:13:19.574156 11702 raft_consensus.cc:374] T 8bebd7fb7deb4e2caa129de3d356205a P a0c7e62d8f73465585819096c63c4a53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:19.574199 11702 raft_consensus.cc:729] T 8bebd7fb7deb4e2caa129de3d356205a P a0c7e62d8f73465585819096c63c4a53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0c7e62d8f73465585819096c63c4a53, State: Initialized, Role: FOLLOWER
I20260811 05:13:19.574360 11702 consensus_queue.cc:260] T 8bebd7fb7deb4e2caa129de3d356205a P a0c7e62d8f73465585819096c63c4a53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } } peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } }
I20260811 05:13:19.574674 11702 ts_tablet_manager.cc:1428] T 8bebd7fb7deb4e2caa129de3d356205a P a0c7e62d8f73465585819096c63c4a53: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 05:13:19.574714 11647 heartbeater.cc:503] Master 127.28.143.190:39863 was elected leader, sending a full tablet report...
I20260811 05:13:19.574930 11703 raft_consensus.cc:348] T 8bebd7fb7deb4e2caa129de3d356205a P 8f7748967c7344fca44f6bfe49a720de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } } peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } }
I20260811 05:13:19.575098 11703 raft_consensus.cc:374] T 8bebd7fb7deb4e2caa129de3d356205a P 8f7748967c7344fca44f6bfe49a720de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:19.575140 11703 raft_consensus.cc:729] T 8bebd7fb7deb4e2caa129de3d356205a P 8f7748967c7344fca44f6bfe49a720de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f7748967c7344fca44f6bfe49a720de, State: Initialized, Role: FOLLOWER
I20260811 05:13:19.575513 11703 consensus_queue.cc:260] T 8bebd7fb7deb4e2caa129de3d356205a P 8f7748967c7344fca44f6bfe49a720de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } } peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } }
I20260811 05:13:19.575595 11706 raft_consensus.cc:348] T 8bebd7fb7deb4e2caa129de3d356205a P 9dbce36fab4e4dd695c4d4d879e301c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } } peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } }
I20260811 05:13:19.575760 11706 raft_consensus.cc:374] T 8bebd7fb7deb4e2caa129de3d356205a P 9dbce36fab4e4dd695c4d4d879e301c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:19.575814 11706 raft_consensus.cc:729] T 8bebd7fb7deb4e2caa129de3d356205a P 9dbce36fab4e4dd695c4d4d879e301c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9dbce36fab4e4dd695c4d4d879e301c5, State: Initialized, Role: FOLLOWER
I20260811 05:13:19.575887 11362 heartbeater.cc:503] Master 127.28.143.190:39863 was elected leader, sending a full tablet report...
I20260811 05:13:19.575920 11703 ts_tablet_manager.cc:1428] T 8bebd7fb7deb4e2caa129de3d356205a P 8f7748967c7344fca44f6bfe49a720de: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 05:13:19.575947 11706 consensus_queue.cc:260] T 8bebd7fb7deb4e2caa129de3d356205a P 9dbce36fab4e4dd695c4d4d879e301c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } } peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } }
I20260811 05:13:19.576205 11706 ts_tablet_manager.cc:1428] T 8bebd7fb7deb4e2caa129de3d356205a P 9dbce36fab4e4dd695c4d4d879e301c5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 05:13:19.576386 11101 heartbeater.cc:503] Master 127.28.143.190:39863 was elected leader, sending a full tablet report...
W20260811 05:13:19.590793 11365 tablet.cc:2369] T 8bebd7fb7deb4e2caa129de3d356205a P 8f7748967c7344fca44f6bfe49a720de: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:13:19.640702 11709 raft_consensus.cc:482] T 8bebd7fb7deb4e2caa129de3d356205a P a0c7e62d8f73465585819096c63c4a53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:13:19.640851 11709 raft_consensus.cc:504] T 8bebd7fb7deb4e2caa129de3d356205a P a0c7e62d8f73465585819096c63c4a53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } } peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } }
I20260811 05:13:19.641191 11709 leader_election.cc:288] T 8bebd7fb7deb4e2caa129de3d356205a P a0c7e62d8f73465585819096c63c4a53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f7748967c7344fca44f6bfe49a720de (127.28.143.130:41671), 9dbce36fab4e4dd695c4d4d879e301c5 (127.28.143.129:37049)
I20260811 05:13:19.644232 11435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:19.641451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:41671 (local address 127.28.143.131:58191)
0811 05:13:19.641560 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:19.641577 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:19.641648 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:19.641785 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:19.641786 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:19.641788 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:19.641859 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:19.641861 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.642544 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.642545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:19.643304 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:19.643306 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.643928 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.643930 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:19.644025 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:19.644029 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:13:19.644082 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 05:13:19.644131 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 05:13:19.644214 11693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:19.641603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:58191 (local address 127.28.143.130:41671)
0811 05:13:19.641655 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:19.641657 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:19.641688 (+ 31us) server_negotiation.cc:407] Connection header received
0811 05:13:19.641718 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:19.641719 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:19.641728 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:19.641773 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:19.641921 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.642442 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.643408 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.643843 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.643916 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:19.643927 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:19.644095 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 05:13:19.644132 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 05:13:19.644380 11701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:19.641704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:42283 (local address 127.28.143.129:37049)
0811 05:13:19.641754 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:19.641756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:19.641808 (+ 52us) server_negotiation.cc:407] Connection header received
0811 05:13:19.641920 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:19.641921 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:19.641928 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:19.641977 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:19.642123 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.642621 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.643654 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.644046 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.644111 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:19.644119 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:19.644274 (+ 155us) server_negotiation.cc:299] Negotiation successful
0811 05:13:19.644313 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 05:13:19.644411 11715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:19.641574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:37049 (local address 127.28.143.131:42283)
0811 05:13:19.641751 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:19.641760 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:19.641823 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:19.641993 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:19.641993 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:19.641995 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:19.642065 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:19.642067 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.642705 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.642706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:19.643483 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:19.643486 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.644125 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.644126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:19.644211 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:19.644214 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 05:13:19.644260 (+ 46us) client_negotiation.cc:240] Negotiation successful
0811 05:13:19.644303 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":111,"threads_started":1}
I20260811 05:13:19.644706 11280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bebd7fb7deb4e2caa129de3d356205a" candidate_uuid: "a0c7e62d8f73465585819096c63c4a53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f7748967c7344fca44f6bfe49a720de" is_pre_election: true
I20260811 05:13:19.644778 11006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bebd7fb7deb4e2caa129de3d356205a" candidate_uuid: "a0c7e62d8f73465585819096c63c4a53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" is_pre_election: true
I20260811 05:13:19.644896 11280 raft_consensus.cc:2454] T 8bebd7fb7deb4e2caa129de3d356205a P 8f7748967c7344fca44f6bfe49a720de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0c7e62d8f73465585819096c63c4a53 in term 0.
I20260811 05:13:19.644948 11006 raft_consensus.cc:2454] T 8bebd7fb7deb4e2caa129de3d356205a P 9dbce36fab4e4dd695c4d4d879e301c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0c7e62d8f73465585819096c63c4a53 in term 0.
I20260811 05:13:19.645185 11426 leader_election.cc:302] T 8bebd7fb7deb4e2caa129de3d356205a P a0c7e62d8f73465585819096c63c4a53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f7748967c7344fca44f6bfe49a720de, a0c7e62d8f73465585819096c63c4a53; no voters:
I20260811 05:13:19.645318 11709 raft_consensus.cc:2781] T 8bebd7fb7deb4e2caa129de3d356205a P a0c7e62d8f73465585819096c63c4a53 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:13:19.645386 11709 raft_consensus.cc:482] T 8bebd7fb7deb4e2caa129de3d356205a P a0c7e62d8f73465585819096c63c4a53 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:13:19.645404 11709 raft_consensus.cc:3037] T 8bebd7fb7deb4e2caa129de3d356205a P a0c7e62d8f73465585819096c63c4a53 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:19.646273 11709 raft_consensus.cc:504] T 8bebd7fb7deb4e2caa129de3d356205a P a0c7e62d8f73465585819096c63c4a53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } } peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } }
I20260811 05:13:19.646413 11709 leader_election.cc:288] T 8bebd7fb7deb4e2caa129de3d356205a P a0c7e62d8f73465585819096c63c4a53 [CANDIDATE]: Term 1 election: Requested vote from peers 8f7748967c7344fca44f6bfe49a720de (127.28.143.130:41671), 9dbce36fab4e4dd695c4d4d879e301c5 (127.28.143.129:37049)
I20260811 05:13:19.646711 11006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bebd7fb7deb4e2caa129de3d356205a" candidate_uuid: "a0c7e62d8f73465585819096c63c4a53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9dbce36fab4e4dd695c4d4d879e301c5"
I20260811 05:13:19.646714 11280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bebd7fb7deb4e2caa129de3d356205a" candidate_uuid: "a0c7e62d8f73465585819096c63c4a53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f7748967c7344fca44f6bfe49a720de"
I20260811 05:13:19.646843 11280 raft_consensus.cc:3037] T 8bebd7fb7deb4e2caa129de3d356205a P 8f7748967c7344fca44f6bfe49a720de [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:19.646844 11006 raft_consensus.cc:3037] T 8bebd7fb7deb4e2caa129de3d356205a P 9dbce36fab4e4dd695c4d4d879e301c5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:19.647796 11006 raft_consensus.cc:2454] T 8bebd7fb7deb4e2caa129de3d356205a P 9dbce36fab4e4dd695c4d4d879e301c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0c7e62d8f73465585819096c63c4a53 in term 1.
I20260811 05:13:19.647796 11280 raft_consensus.cc:2454] T 8bebd7fb7deb4e2caa129de3d356205a P 8f7748967c7344fca44f6bfe49a720de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0c7e62d8f73465585819096c63c4a53 in term 1.
I20260811 05:13:19.648044 11426 leader_election.cc:302] T 8bebd7fb7deb4e2caa129de3d356205a P a0c7e62d8f73465585819096c63c4a53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9dbce36fab4e4dd695c4d4d879e301c5, a0c7e62d8f73465585819096c63c4a53; no voters:
I20260811 05:13:19.648144 11709 raft_consensus.cc:2781] T 8bebd7fb7deb4e2caa129de3d356205a P a0c7e62d8f73465585819096c63c4a53 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:19.648365 11709 raft_consensus.cc:686] T 8bebd7fb7deb4e2caa129de3d356205a P a0c7e62d8f73465585819096c63c4a53 [term 1 LEADER]: Becoming Leader. State: Replica: a0c7e62d8f73465585819096c63c4a53, State: Running, Role: LEADER
I20260811 05:13:19.648514 11709 consensus_queue.cc:237] T 8bebd7fb7deb4e2caa129de3d356205a P a0c7e62d8f73465585819096c63c4a53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } } peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } }
I20260811 05:13:19.649609 10695 catalog_manager.cc:5495] T 8bebd7fb7deb4e2caa129de3d356205a P a0c7e62d8f73465585819096c63c4a53 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0c7e62d8f73465585819096c63c4a53 (127.28.143.131). New cstate: current_term: 1 leader_uuid: "a0c7e62d8f73465585819096c63c4a53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } health_report { overall_health: UNKNOWN } } }
W20260811 05:13:19.658315 11102 tablet.cc:2369] T 8bebd7fb7deb4e2caa129de3d356205a P 9dbce36fab4e4dd695c4d4d879e301c5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:13:19.659741 11694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:19.657372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36156 (local address 127.28.143.131:40121)
0811 05:13:19.657425 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:19.657426 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:19.657432 (+ 6us) server_negotiation.cc:407] Connection header received
0811 05:13:19.657463 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:19.657464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:19.657481 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:19.657542 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:13:19.657677 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.658170 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.658971 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.659158 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.659213 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:19.659404 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:13:19.659516 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:13:19.659577 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:19.659656 (+ 79us) server_negotiation.cc:299] Negotiation successful
0811 05:13:19.659676 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 05:13:19.662034 11006 raft_consensus.cc:1261] T 8bebd7fb7deb4e2caa129de3d356205a P 9dbce36fab4e4dd695c4d4d879e301c5 [term 1 FOLLOWER]: Refusing update from remote peer a0c7e62d8f73465585819096c63c4a53: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:19.662060 11280 raft_consensus.cc:1261] T 8bebd7fb7deb4e2caa129de3d356205a P 8f7748967c7344fca44f6bfe49a720de [term 1 FOLLOWER]: Refusing update from remote peer a0c7e62d8f73465585819096c63c4a53: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:19.662338 11716 consensus_queue.cc:1035] T 8bebd7fb7deb4e2caa129de3d356205a P a0c7e62d8f73465585819096c63c4a53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:19.662478 11709 consensus_queue.cc:1035] T 8bebd7fb7deb4e2caa129de3d356205a P a0c7e62d8f73465585819096c63c4a53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:19.665159 11723 mvcc.cc:205] Tried to move back new op lower bound from 7317197617812934656 to 7317197617761226752. Current Snapshot: MvccSnapshot[applied={T|T < 7317197617812934656}]
I20260811 05:13:19.665760 11727 mvcc.cc:205] Tried to move back new op lower bound from 7317197617812934656 to 7317197617761226752. Current Snapshot: MvccSnapshot[applied={T|T < 7317197617812934656}]
I20260811 05:13:19.666706 11725 mvcc.cc:205] Tried to move back new op lower bound from 7317197617812934656 to 7317197617761226752. Current Snapshot: MvccSnapshot[applied={T|T < 7317197617812934656}]
I20260811 05:13:19.667948 10694 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:19.668066 10694 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:19.668929 10694 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4b9d89233cf04d7697a4dd645a3893ae: Sending DeleteTablet for 3 replicas of tablet 8bebd7fb7deb4e2caa129de3d356205a
I20260811 05:13:19.669319 11518 tablet_service.cc:1514] Processing DeleteTablet for tablet 8bebd7fb7deb4e2caa129de3d356205a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:19 UTC) from {username='oryx', principal='oryx/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:36154
I20260811 05:13:19.669373 10967 tablet_service.cc:1514] Processing DeleteTablet for tablet 8bebd7fb7deb4e2caa129de3d356205a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:19 UTC) from {username='oryx', principal='oryx/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:48010
I20260811 05:13:19.669435 11246 tablet_service.cc:1514] Processing DeleteTablet for tablet 8bebd7fb7deb4e2caa129de3d356205a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:19 UTC) from {username='oryx', principal='oryx/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:42342
I20260811 05:13:19.669613 10694 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57556:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:13:19.669766 10694 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:13:19.669858 11742 tablet_replica.cc:330] T 8bebd7fb7deb4e2caa129de3d356205a P 8f7748967c7344fca44f6bfe49a720de: stopping tablet replica
I20260811 05:13:19.669978 11742 raft_consensus.cc:2229] T 8bebd7fb7deb4e2caa129de3d356205a P 8f7748967c7344fca44f6bfe49a720de [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:19.670099 11742 raft_consensus.cc:2258] T 8bebd7fb7deb4e2caa129de3d356205a P 8f7748967c7344fca44f6bfe49a720de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:19.671150 10967 tablet_service.cc:1467] Processing CreateTablet for tablet 0957b1f1b28b4980ac70ab15307466bc (DEFAULT_TABLE table=test-table [id=0a6b329cb65d48a59c1bd43b007f4bed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:19.671306 10967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0957b1f1b28b4980ac70ab15307466bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:19.671293 11246 tablet_service.cc:1467] Processing CreateTablet for tablet 0957b1f1b28b4980ac70ab15307466bc (DEFAULT_TABLE table=test-table [id=0a6b329cb65d48a59c1bd43b007f4bed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:19.671402 11246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0957b1f1b28b4980ac70ab15307466bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:19.671646 11742 ts_tablet_manager.cc:1905] T 8bebd7fb7deb4e2caa129de3d356205a P 8f7748967c7344fca44f6bfe49a720de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:19.672744 11740 tablet_replica.cc:330] T 8bebd7fb7deb4e2caa129de3d356205a P a0c7e62d8f73465585819096c63c4a53: stopping tablet replica
I20260811 05:13:19.672740 11706 tablet_bootstrap.cc:492] T 0957b1f1b28b4980ac70ab15307466bc P 9dbce36fab4e4dd695c4d4d879e301c5: Bootstrap starting.
I20260811 05:13:19.672852 11740 raft_consensus.cc:2229] T 8bebd7fb7deb4e2caa129de3d356205a P a0c7e62d8f73465585819096c63c4a53 [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:13:19.672891 11703 tablet_bootstrap.cc:492] T 0957b1f1b28b4980ac70ab15307466bc P 8f7748967c7344fca44f6bfe49a720de: Bootstrap starting.
I20260811 05:13:19.672966 11740 raft_consensus.cc:2258] T 8bebd7fb7deb4e2caa129de3d356205a P a0c7e62d8f73465585819096c63c4a53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:19.673108 11741 tablet_replica.cc:330] T 8bebd7fb7deb4e2caa129de3d356205a P 9dbce36fab4e4dd695c4d4d879e301c5: stopping tablet replica
I20260811 05:13:19.673152 11706 tablet_bootstrap.cc:654] T 0957b1f1b28b4980ac70ab15307466bc P 9dbce36fab4e4dd695c4d4d879e301c5: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:19.673211 11741 raft_consensus.cc:2229] T 8bebd7fb7deb4e2caa129de3d356205a P 9dbce36fab4e4dd695c4d4d879e301c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:19.673300 11703 tablet_bootstrap.cc:654] T 0957b1f1b28b4980ac70ab15307466bc P 8f7748967c7344fca44f6bfe49a720de: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:19.673323 11741 raft_consensus.cc:2258] T 8bebd7fb7deb4e2caa129de3d356205a P 9dbce36fab4e4dd695c4d4d879e301c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:19.673529 11518 tablet_service.cc:1467] Processing CreateTablet for tablet 0957b1f1b28b4980ac70ab15307466bc (DEFAULT_TABLE table=test-table [id=0a6b329cb65d48a59c1bd43b007f4bed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:19.673635 11518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0957b1f1b28b4980ac70ab15307466bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:19.674062 11741 ts_tablet_manager.cc:1905] T 8bebd7fb7deb4e2caa129de3d356205a P 9dbce36fab4e4dd695c4d4d879e301c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:19.674201 11742 ts_tablet_manager.cc:1918] T 8bebd7fb7deb4e2caa129de3d356205a P 8f7748967c7344fca44f6bfe49a720de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:13:19.674288 11742 log.cc:1196] T 8bebd7fb7deb4e2caa129de3d356205a P 8f7748967c7344fca44f6bfe49a720de: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-1/wal/wals/8bebd7fb7deb4e2caa129de3d356205a
I20260811 05:13:19.674567 11742 ts_tablet_manager.cc:1939] T 8bebd7fb7deb4e2caa129de3d356205a P 8f7748967c7344fca44f6bfe49a720de: Deleting consensus metadata
I20260811 05:13:19.674984 11702 tablet_bootstrap.cc:492] T 0957b1f1b28b4980ac70ab15307466bc P a0c7e62d8f73465585819096c63c4a53: Bootstrap starting.
I20260811 05:13:19.675606 11702 tablet_bootstrap.cc:654] T 0957b1f1b28b4980ac70ab15307466bc P a0c7e62d8f73465585819096c63c4a53: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:19.676343 11702 tablet_bootstrap.cc:492] T 0957b1f1b28b4980ac70ab15307466bc P a0c7e62d8f73465585819096c63c4a53: No bootstrap required, opened a new log
I20260811 05:13:19.676402 11702 ts_tablet_manager.cc:1397] T 0957b1f1b28b4980ac70ab15307466bc P a0c7e62d8f73465585819096c63c4a53: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:19.676561 11702 raft_consensus.cc:348] T 0957b1f1b28b4980ac70ab15307466bc P a0c7e62d8f73465585819096c63c4a53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } }
I20260811 05:13:19.676684 11702 raft_consensus.cc:374] T 0957b1f1b28b4980ac70ab15307466bc P a0c7e62d8f73465585819096c63c4a53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:19.676708 11702 raft_consensus.cc:729] T 0957b1f1b28b4980ac70ab15307466bc P a0c7e62d8f73465585819096c63c4a53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0c7e62d8f73465585819096c63c4a53, State: Initialized, Role: FOLLOWER
I20260811 05:13:19.676760 11702 consensus_queue.cc:260] T 0957b1f1b28b4980ac70ab15307466bc P a0c7e62d8f73465585819096c63c4a53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } }
I20260811 05:13:19.676895 11702 ts_tablet_manager.cc:1428] T 0957b1f1b28b4980ac70ab15307466bc P a0c7e62d8f73465585819096c63c4a53: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:19.676964 11740 ts_tablet_manager.cc:1905] T 8bebd7fb7deb4e2caa129de3d356205a P a0c7e62d8f73465585819096c63c4a53: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:19.677191 11741 ts_tablet_manager.cc:1918] T 8bebd7fb7deb4e2caa129de3d356205a P 9dbce36fab4e4dd695c4d4d879e301c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:13:19.677320 11741 log.cc:1196] T 8bebd7fb7deb4e2caa129de3d356205a P 9dbce36fab4e4dd695c4d4d879e301c5: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-0/wal/wals/8bebd7fb7deb4e2caa129de3d356205a
I20260811 05:13:19.677742 11741 ts_tablet_manager.cc:1939] T 8bebd7fb7deb4e2caa129de3d356205a P 9dbce36fab4e4dd695c4d4d879e301c5: Deleting consensus metadata
I20260811 05:13:19.678308 10674 catalog_manager.cc:4841] TS 9dbce36fab4e4dd695c4d4d879e301c5 (127.28.143.129:37049): tablet 8bebd7fb7deb4e2caa129de3d356205a (table test-table [id=8107bd8315d64bf39610c87ef8ea5bd5]) successfully deleted
I20260811 05:13:19.678534 11740 ts_tablet_manager.cc:1918] T 8bebd7fb7deb4e2caa129de3d356205a P a0c7e62d8f73465585819096c63c4a53: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:13:19.678596 11740 log.cc:1196] T 8bebd7fb7deb4e2caa129de3d356205a P a0c7e62d8f73465585819096c63c4a53: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-2/wal/wals/8bebd7fb7deb4e2caa129de3d356205a
I20260811 05:13:19.678777 11740 ts_tablet_manager.cc:1939] T 8bebd7fb7deb4e2caa129de3d356205a P a0c7e62d8f73465585819096c63c4a53: Deleting consensus metadata
I20260811 05:13:19.679211 10674 catalog_manager.cc:4841] TS 8f7748967c7344fca44f6bfe49a720de (127.28.143.130:41671): tablet 8bebd7fb7deb4e2caa129de3d356205a (table test-table [id=8107bd8315d64bf39610c87ef8ea5bd5]) successfully deleted
I20260811 05:13:19.679322 10674 catalog_manager.cc:4841] TS a0c7e62d8f73465585819096c63c4a53 (127.28.143.131:40121): tablet 8bebd7fb7deb4e2caa129de3d356205a (table test-table [id=8107bd8315d64bf39610c87ef8ea5bd5]) successfully deleted
I20260811 05:13:19.681025 11703 tablet_bootstrap.cc:492] T 0957b1f1b28b4980ac70ab15307466bc P 8f7748967c7344fca44f6bfe49a720de: No bootstrap required, opened a new log
I20260811 05:13:19.681087 11703 ts_tablet_manager.cc:1397] T 0957b1f1b28b4980ac70ab15307466bc P 8f7748967c7344fca44f6bfe49a720de: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260811 05:13:19.681210 11706 tablet_bootstrap.cc:492] T 0957b1f1b28b4980ac70ab15307466bc P 9dbce36fab4e4dd695c4d4d879e301c5: No bootstrap required, opened a new log
I20260811 05:13:19.681269 11706 ts_tablet_manager.cc:1397] T 0957b1f1b28b4980ac70ab15307466bc P 9dbce36fab4e4dd695c4d4d879e301c5: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20260811 05:13:19.681264 11703 raft_consensus.cc:348] T 0957b1f1b28b4980ac70ab15307466bc P 8f7748967c7344fca44f6bfe49a720de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } }
I20260811 05:13:19.681352 11703 raft_consensus.cc:374] T 0957b1f1b28b4980ac70ab15307466bc P 8f7748967c7344fca44f6bfe49a720de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:19.681380 11703 raft_consensus.cc:729] T 0957b1f1b28b4980ac70ab15307466bc P 8f7748967c7344fca44f6bfe49a720de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f7748967c7344fca44f6bfe49a720de, State: Initialized, Role: FOLLOWER
I20260811 05:13:19.681427 11703 consensus_queue.cc:260] T 0957b1f1b28b4980ac70ab15307466bc P 8f7748967c7344fca44f6bfe49a720de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } }
I20260811 05:13:19.681427 11706 raft_consensus.cc:348] T 0957b1f1b28b4980ac70ab15307466bc P 9dbce36fab4e4dd695c4d4d879e301c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } }
I20260811 05:13:19.681495 11706 raft_consensus.cc:374] T 0957b1f1b28b4980ac70ab15307466bc P 9dbce36fab4e4dd695c4d4d879e301c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:19.681514 11706 raft_consensus.cc:729] T 0957b1f1b28b4980ac70ab15307466bc P 9dbce36fab4e4dd695c4d4d879e301c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9dbce36fab4e4dd695c4d4d879e301c5, State: Initialized, Role: FOLLOWER
I20260811 05:13:19.681524 11703 ts_tablet_manager.cc:1428] T 0957b1f1b28b4980ac70ab15307466bc P 8f7748967c7344fca44f6bfe49a720de: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 05:13:19.681576 11706 consensus_queue.cc:260] T 0957b1f1b28b4980ac70ab15307466bc P 9dbce36fab4e4dd695c4d4d879e301c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } }
I20260811 05:13:19.681676 11706 ts_tablet_manager.cc:1428] T 0957b1f1b28b4980ac70ab15307466bc P 9dbce36fab4e4dd695c4d4d879e301c5: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 05:13:19.758313 11709 raft_consensus.cc:482] T 0957b1f1b28b4980ac70ab15307466bc P a0c7e62d8f73465585819096c63c4a53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:13:19.758461 11709 raft_consensus.cc:504] T 0957b1f1b28b4980ac70ab15307466bc P a0c7e62d8f73465585819096c63c4a53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } }
I20260811 05:13:19.758602 11709 leader_election.cc:288] T 0957b1f1b28b4980ac70ab15307466bc P a0c7e62d8f73465585819096c63c4a53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9dbce36fab4e4dd695c4d4d879e301c5 (127.28.143.129:37049), 8f7748967c7344fca44f6bfe49a720de (127.28.143.130:41671)
I20260811 05:13:19.758932 11006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0957b1f1b28b4980ac70ab15307466bc" candidate_uuid: "a0c7e62d8f73465585819096c63c4a53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" is_pre_election: true
I20260811 05:13:19.758930 11280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0957b1f1b28b4980ac70ab15307466bc" candidate_uuid: "a0c7e62d8f73465585819096c63c4a53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f7748967c7344fca44f6bfe49a720de" is_pre_election: true
I20260811 05:13:19.759055 11006 raft_consensus.cc:2454] T 0957b1f1b28b4980ac70ab15307466bc P 9dbce36fab4e4dd695c4d4d879e301c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0c7e62d8f73465585819096c63c4a53 in term 0.
I20260811 05:13:19.759063 11280 raft_consensus.cc:2454] T 0957b1f1b28b4980ac70ab15307466bc P 8f7748967c7344fca44f6bfe49a720de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0c7e62d8f73465585819096c63c4a53 in term 0.
I20260811 05:13:19.759239 11426 leader_election.cc:302] T 0957b1f1b28b4980ac70ab15307466bc P a0c7e62d8f73465585819096c63c4a53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f7748967c7344fca44f6bfe49a720de, a0c7e62d8f73465585819096c63c4a53; no voters:
I20260811 05:13:19.759362 11709 raft_consensus.cc:2781] T 0957b1f1b28b4980ac70ab15307466bc P a0c7e62d8f73465585819096c63c4a53 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:13:19.759418 11709 raft_consensus.cc:482] T 0957b1f1b28b4980ac70ab15307466bc P a0c7e62d8f73465585819096c63c4a53 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:13:19.759439 11709 raft_consensus.cc:3037] T 0957b1f1b28b4980ac70ab15307466bc P a0c7e62d8f73465585819096c63c4a53 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:19.760154 11709 raft_consensus.cc:504] T 0957b1f1b28b4980ac70ab15307466bc P a0c7e62d8f73465585819096c63c4a53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } }
I20260811 05:13:19.760279 11709 leader_election.cc:288] T 0957b1f1b28b4980ac70ab15307466bc P a0c7e62d8f73465585819096c63c4a53 [CANDIDATE]: Term 1 election: Requested vote from peers 9dbce36fab4e4dd695c4d4d879e301c5 (127.28.143.129:37049), 8f7748967c7344fca44f6bfe49a720de (127.28.143.130:41671)
I20260811 05:13:19.760540 11006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0957b1f1b28b4980ac70ab15307466bc" candidate_uuid: "a0c7e62d8f73465585819096c63c4a53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9dbce36fab4e4dd695c4d4d879e301c5"
I20260811 05:13:19.760563 11280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0957b1f1b28b4980ac70ab15307466bc" candidate_uuid: "a0c7e62d8f73465585819096c63c4a53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f7748967c7344fca44f6bfe49a720de"
I20260811 05:13:19.760661 11280 raft_consensus.cc:3037] T 0957b1f1b28b4980ac70ab15307466bc P 8f7748967c7344fca44f6bfe49a720de [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:19.760679 11006 raft_consensus.cc:3037] T 0957b1f1b28b4980ac70ab15307466bc P 9dbce36fab4e4dd695c4d4d879e301c5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:19.761329 11280 raft_consensus.cc:2454] T 0957b1f1b28b4980ac70ab15307466bc P 8f7748967c7344fca44f6bfe49a720de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0c7e62d8f73465585819096c63c4a53 in term 1.
I20260811 05:13:19.761528 11426 leader_election.cc:302] T 0957b1f1b28b4980ac70ab15307466bc P a0c7e62d8f73465585819096c63c4a53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f7748967c7344fca44f6bfe49a720de, a0c7e62d8f73465585819096c63c4a53; no voters:
I20260811 05:13:19.761627 11709 raft_consensus.cc:2781] T 0957b1f1b28b4980ac70ab15307466bc P a0c7e62d8f73465585819096c63c4a53 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:19.761628 11006 raft_consensus.cc:2454] T 0957b1f1b28b4980ac70ab15307466bc P 9dbce36fab4e4dd695c4d4d879e301c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0c7e62d8f73465585819096c63c4a53 in term 1.
I20260811 05:13:19.761672 11709 raft_consensus.cc:686] T 0957b1f1b28b4980ac70ab15307466bc P a0c7e62d8f73465585819096c63c4a53 [term 1 LEADER]: Becoming Leader. State: Replica: a0c7e62d8f73465585819096c63c4a53, State: Running, Role: LEADER
I20260811 05:13:19.761739 11709 consensus_queue.cc:237] T 0957b1f1b28b4980ac70ab15307466bc P a0c7e62d8f73465585819096c63c4a53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } }
W20260811 05:13:19.761950 11648 tablet.cc:2369] T 0957b1f1b28b4980ac70ab15307466bc P a0c7e62d8f73465585819096c63c4a53: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:13:19.762463 10695 catalog_manager.cc:5495] T 0957b1f1b28b4980ac70ab15307466bc P a0c7e62d8f73465585819096c63c4a53 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0c7e62d8f73465585819096c63c4a53 (127.28.143.131). New cstate: current_term: 1 leader_uuid: "a0c7e62d8f73465585819096c63c4a53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } health_report { overall_health: HEALTHY } } }
Aug 11 05:13:19 dist-test-slave-f7th krb5kdc[10615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425198, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.143.190@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM
I20260811 05:13:19.793471 11762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:19.785517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39863 (local address 127.0.0.1:57564)
0811 05:13:19.785703 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:19.785716 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:19.785811 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:19.785960 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:19.785961 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:19.786440 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:19.786514 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:19.786517 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.787298 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.787299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:19.788046 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:19.788049 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.788420 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.788421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:19.788476 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:19.789154 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:19.789159 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:19.790774 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:19.792168 (+ 1394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:19.792177 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:19.792179 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:19.792400 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:19.792714 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:19.792715 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:19.792715 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:19.792787 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:19.793049 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:19.793060 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:19.793108 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 05:13:19.793167 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 05:13:19.793308 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":93,"threads_started":1}
I20260811 05:13:19.793546 10790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:19.785626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57564 (local address 127.28.143.190:39863)
0811 05:13:19.785684 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:19.785686 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:19.785823 (+ 137us) server_negotiation.cc:407] Connection header received
0811 05:13:19.785859 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:19.785860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:19.785871 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:19.785934 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:19.786667 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.787209 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.788153 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:19.788342 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:19.788408 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:19.790867 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:19.790868 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:19.790869 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:19.790878 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:19.792075 (+ 1197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:19.792495 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:19.792496 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:19.792496 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:19.792586 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:19.792842 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:19.792843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:19.792843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:19.792984 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:19.793044 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:19.793217 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 05:13:19.793386 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":75,"server-negotiator.queue_time_us":33}
I20260811 05:13:19.795982 10695 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:57564:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 05:13:19.796242 10695 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:13:19.797989 11518 tablet_service.cc:1467] Processing CreateTablet for tablet 6779fc4ade994e5f88a0aa56045d7f12 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6c253fc446644564a8b5a4fdc1053e66]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 05:13:19.798003 11246 tablet_service.cc:1467] Processing CreateTablet for tablet 6779fc4ade994e5f88a0aa56045d7f12 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6c253fc446644564a8b5a4fdc1053e66]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 05:13:19.797979 10967 tablet_service.cc:1467] Processing CreateTablet for tablet 6779fc4ade994e5f88a0aa56045d7f12 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6c253fc446644564a8b5a4fdc1053e66]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 05:13:19.798178 11246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6779fc4ade994e5f88a0aa56045d7f12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:19.798180 10967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6779fc4ade994e5f88a0aa56045d7f12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:19.798179 11518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6779fc4ade994e5f88a0aa56045d7f12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:19.799486 11702 tablet_bootstrap.cc:492] T 6779fc4ade994e5f88a0aa56045d7f12 P a0c7e62d8f73465585819096c63c4a53: Bootstrap starting.
I20260811 05:13:19.799793 11703 tablet_bootstrap.cc:492] T 6779fc4ade994e5f88a0aa56045d7f12 P 8f7748967c7344fca44f6bfe49a720de: Bootstrap starting.
I20260811 05:13:19.799804 11706 tablet_bootstrap.cc:492] T 6779fc4ade994e5f88a0aa56045d7f12 P 9dbce36fab4e4dd695c4d4d879e301c5: Bootstrap starting.
I20260811 05:13:19.799926 11702 tablet_bootstrap.cc:654] T 6779fc4ade994e5f88a0aa56045d7f12 P a0c7e62d8f73465585819096c63c4a53: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:19.800175 11703 tablet_bootstrap.cc:654] T 6779fc4ade994e5f88a0aa56045d7f12 P 8f7748967c7344fca44f6bfe49a720de: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:19.800251 11706 tablet_bootstrap.cc:654] T 6779fc4ade994e5f88a0aa56045d7f12 P 9dbce36fab4e4dd695c4d4d879e301c5: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:19.800900 11702 tablet_bootstrap.cc:492] T 6779fc4ade994e5f88a0aa56045d7f12 P a0c7e62d8f73465585819096c63c4a53: No bootstrap required, opened a new log
I20260811 05:13:19.800976 11702 ts_tablet_manager.cc:1397] T 6779fc4ade994e5f88a0aa56045d7f12 P a0c7e62d8f73465585819096c63c4a53: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 05:13:19.801291 11706 tablet_bootstrap.cc:492] T 6779fc4ade994e5f88a0aa56045d7f12 P 9dbce36fab4e4dd695c4d4d879e301c5: No bootstrap required, opened a new log
I20260811 05:13:19.801304 11702 raft_consensus.cc:348] T 6779fc4ade994e5f88a0aa56045d7f12 P a0c7e62d8f73465585819096c63c4a53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } }
I20260811 05:13:19.801363 11706 ts_tablet_manager.cc:1397] T 6779fc4ade994e5f88a0aa56045d7f12 P 9dbce36fab4e4dd695c4d4d879e301c5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 05:13:19.801393 11702 raft_consensus.cc:374] T 6779fc4ade994e5f88a0aa56045d7f12 P a0c7e62d8f73465585819096c63c4a53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:19.801415 11702 raft_consensus.cc:729] T 6779fc4ade994e5f88a0aa56045d7f12 P a0c7e62d8f73465585819096c63c4a53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0c7e62d8f73465585819096c63c4a53, State: Initialized, Role: FOLLOWER
I20260811 05:13:19.801498 11702 consensus_queue.cc:260] T 6779fc4ade994e5f88a0aa56045d7f12 P a0c7e62d8f73465585819096c63c4a53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } }
I20260811 05:13:19.801517 11706 raft_consensus.cc:348] T 6779fc4ade994e5f88a0aa56045d7f12 P 9dbce36fab4e4dd695c4d4d879e301c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } }
I20260811 05:13:19.801584 11706 raft_consensus.cc:374] T 6779fc4ade994e5f88a0aa56045d7f12 P 9dbce36fab4e4dd695c4d4d879e301c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:19.801602 11706 raft_consensus.cc:729] T 6779fc4ade994e5f88a0aa56045d7f12 P 9dbce36fab4e4dd695c4d4d879e301c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9dbce36fab4e4dd695c4d4d879e301c5, State: Initialized, Role: FOLLOWER
I20260811 05:13:19.801616 11702 ts_tablet_manager.cc:1428] T 6779fc4ade994e5f88a0aa56045d7f12 P a0c7e62d8f73465585819096c63c4a53: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 05:13:19.801645 11706 consensus_queue.cc:260] T 6779fc4ade994e5f88a0aa56045d7f12 P 9dbce36fab4e4dd695c4d4d879e301c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } }
I20260811 05:13:19.801748 11706 ts_tablet_manager.cc:1428] T 6779fc4ade994e5f88a0aa56045d7f12 P 9dbce36fab4e4dd695c4d4d879e301c5: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 05:13:19.801757 11716 tablet_replica.cc:439] T 6779fc4ade994e5f88a0aa56045d7f12 P a0c7e62d8f73465585819096c63c4a53: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } } }
I20260811 05:13:19.801877 11716 tablet_replica.cc:442] T 6779fc4ade994e5f88a0aa56045d7f12 P a0c7e62d8f73465585819096c63c4a53: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:13:19.801949 11713 tablet_replica.cc:439] T 6779fc4ade994e5f88a0aa56045d7f12 P 9dbce36fab4e4dd695c4d4d879e301c5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } } }
I20260811 05:13:19.802042 11713 tablet_replica.cc:442] T 6779fc4ade994e5f88a0aa56045d7f12 P 9dbce36fab4e4dd695c4d4d879e301c5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:13:19.802258 11703 tablet_bootstrap.cc:492] T 6779fc4ade994e5f88a0aa56045d7f12 P 8f7748967c7344fca44f6bfe49a720de: No bootstrap required, opened a new log
I20260811 05:13:19.802326 11703 ts_tablet_manager.cc:1397] T 6779fc4ade994e5f88a0aa56045d7f12 P 8f7748967c7344fca44f6bfe49a720de: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260811 05:13:19.802556 11703 raft_consensus.cc:348] T 6779fc4ade994e5f88a0aa56045d7f12 P 8f7748967c7344fca44f6bfe49a720de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } }
I20260811 05:13:19.802645 11703 raft_consensus.cc:374] T 6779fc4ade994e5f88a0aa56045d7f12 P 8f7748967c7344fca44f6bfe49a720de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:19.802668 11703 raft_consensus.cc:729] T 6779fc4ade994e5f88a0aa56045d7f12 P 8f7748967c7344fca44f6bfe49a720de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f7748967c7344fca44f6bfe49a720de, State: Initialized, Role: FOLLOWER
I20260811 05:13:19.802721 11703 consensus_queue.cc:260] T 6779fc4ade994e5f88a0aa56045d7f12 P 8f7748967c7344fca44f6bfe49a720de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } }
I20260811 05:13:19.802831 11703 ts_tablet_manager.cc:1428] T 6779fc4ade994e5f88a0aa56045d7f12 P 8f7748967c7344fca44f6bfe49a720de: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 05:13:19.803023 11711 tablet_replica.cc:439] T 6779fc4ade994e5f88a0aa56045d7f12 P 8f7748967c7344fca44f6bfe49a720de: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } } }
I20260811 05:13:19.803126 11711 tablet_replica.cc:442] T 6779fc4ade994e5f88a0aa56045d7f12 P 8f7748967c7344fca44f6bfe49a720de: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:13:20.018160 11716 raft_consensus.cc:482] T 6779fc4ade994e5f88a0aa56045d7f12 P a0c7e62d8f73465585819096c63c4a53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:13:20.018285 11716 raft_consensus.cc:504] T 6779fc4ade994e5f88a0aa56045d7f12 P a0c7e62d8f73465585819096c63c4a53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } }
I20260811 05:13:20.018489 11716 leader_election.cc:288] T 6779fc4ade994e5f88a0aa56045d7f12 P a0c7e62d8f73465585819096c63c4a53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9dbce36fab4e4dd695c4d4d879e301c5 (127.28.143.129:37049), 8f7748967c7344fca44f6bfe49a720de (127.28.143.130:41671)
I20260811 05:13:20.018795 11006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6779fc4ade994e5f88a0aa56045d7f12" candidate_uuid: "a0c7e62d8f73465585819096c63c4a53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" is_pre_election: true
I20260811 05:13:20.018787 11280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6779fc4ade994e5f88a0aa56045d7f12" candidate_uuid: "a0c7e62d8f73465585819096c63c4a53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f7748967c7344fca44f6bfe49a720de" is_pre_election: true
I20260811 05:13:20.018898 11006 raft_consensus.cc:2454] T 6779fc4ade994e5f88a0aa56045d7f12 P 9dbce36fab4e4dd695c4d4d879e301c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0c7e62d8f73465585819096c63c4a53 in term 0.
I20260811 05:13:20.018899 11280 raft_consensus.cc:2454] T 6779fc4ade994e5f88a0aa56045d7f12 P 8f7748967c7344fca44f6bfe49a720de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0c7e62d8f73465585819096c63c4a53 in term 0.
I20260811 05:13:20.019052 11426 leader_election.cc:302] T 6779fc4ade994e5f88a0aa56045d7f12 P a0c7e62d8f73465585819096c63c4a53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9dbce36fab4e4dd695c4d4d879e301c5, a0c7e62d8f73465585819096c63c4a53; no voters:
I20260811 05:13:20.019156 11716 raft_consensus.cc:2781] T 6779fc4ade994e5f88a0aa56045d7f12 P a0c7e62d8f73465585819096c63c4a53 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:13:20.019203 11716 raft_consensus.cc:482] T 6779fc4ade994e5f88a0aa56045d7f12 P a0c7e62d8f73465585819096c63c4a53 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:13:20.019217 11716 raft_consensus.cc:3037] T 6779fc4ade994e5f88a0aa56045d7f12 P a0c7e62d8f73465585819096c63c4a53 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:20.019950 11716 raft_consensus.cc:504] T 6779fc4ade994e5f88a0aa56045d7f12 P a0c7e62d8f73465585819096c63c4a53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } }
I20260811 05:13:20.020072 11716 leader_election.cc:288] T 6779fc4ade994e5f88a0aa56045d7f12 P a0c7e62d8f73465585819096c63c4a53 [CANDIDATE]: Term 1 election: Requested vote from peers 9dbce36fab4e4dd695c4d4d879e301c5 (127.28.143.129:37049), 8f7748967c7344fca44f6bfe49a720de (127.28.143.130:41671)
I20260811 05:13:20.020253 11006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6779fc4ade994e5f88a0aa56045d7f12" candidate_uuid: "a0c7e62d8f73465585819096c63c4a53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9dbce36fab4e4dd695c4d4d879e301c5"
I20260811 05:13:20.020270 11280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6779fc4ade994e5f88a0aa56045d7f12" candidate_uuid: "a0c7e62d8f73465585819096c63c4a53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f7748967c7344fca44f6bfe49a720de"
I20260811 05:13:20.020336 11280 raft_consensus.cc:3037] T 6779fc4ade994e5f88a0aa56045d7f12 P 8f7748967c7344fca44f6bfe49a720de [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:20.020335 11006 raft_consensus.cc:3037] T 6779fc4ade994e5f88a0aa56045d7f12 P 9dbce36fab4e4dd695c4d4d879e301c5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:20.021026 11280 raft_consensus.cc:2454] T 6779fc4ade994e5f88a0aa56045d7f12 P 8f7748967c7344fca44f6bfe49a720de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0c7e62d8f73465585819096c63c4a53 in term 1.
I20260811 05:13:20.021118 11006 raft_consensus.cc:2454] T 6779fc4ade994e5f88a0aa56045d7f12 P 9dbce36fab4e4dd695c4d4d879e301c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0c7e62d8f73465585819096c63c4a53 in term 1.
I20260811 05:13:20.021214 11426 leader_election.cc:302] T 6779fc4ade994e5f88a0aa56045d7f12 P a0c7e62d8f73465585819096c63c4a53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f7748967c7344fca44f6bfe49a720de, a0c7e62d8f73465585819096c63c4a53; no voters:
I20260811 05:13:20.021308 11716 raft_consensus.cc:2781] T 6779fc4ade994e5f88a0aa56045d7f12 P a0c7e62d8f73465585819096c63c4a53 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:20.021363 11716 raft_consensus.cc:686] T 6779fc4ade994e5f88a0aa56045d7f12 P a0c7e62d8f73465585819096c63c4a53 [term 1 LEADER]: Becoming Leader. State: Replica: a0c7e62d8f73465585819096c63c4a53, State: Running, Role: LEADER
I20260811 05:13:20.021413 11716 consensus_queue.cc:237] T 6779fc4ade994e5f88a0aa56045d7f12 P a0c7e62d8f73465585819096c63c4a53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } }
I20260811 05:13:20.021683 11709 tablet_replica.cc:439] T 6779fc4ade994e5f88a0aa56045d7f12 P a0c7e62d8f73465585819096c63c4a53: TxnStatusTablet state changed. Reason: New leader a0c7e62d8f73465585819096c63c4a53. Latest consensus state: current_term: 1 leader_uuid: "a0c7e62d8f73465585819096c63c4a53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } } }
I20260811 05:13:20.021754 11709 tablet_replica.cc:442] T 6779fc4ade994e5f88a0aa56045d7f12 P a0c7e62d8f73465585819096c63c4a53: This TxnStatusTablet replica's current role is: LEADER
I20260811 05:13:20.021880 11770 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 05:13:20.021993 11770 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 05:13:20.022086 10695 catalog_manager.cc:5495] T 6779fc4ade994e5f88a0aa56045d7f12 P a0c7e62d8f73465585819096c63c4a53 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0c7e62d8f73465585819096c63c4a53 (127.28.143.131). New cstate: current_term: 1 leader_uuid: "a0c7e62d8f73465585819096c63c4a53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } health_report { overall_health: HEALTHY } } }
I20260811 05:13:20.034662 11762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:20.031857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.131:40121 (local address 127.0.0.1:36166)
0811 05:13:20.031930 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:20.031943 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:20.031997 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:20.032183 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:20.032184 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:20.032185 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 05:13:20.032233 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:20.032235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:20.033171 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:20.033172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:20.033791 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:20.033794 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:20.034258 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:20.034259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:20.034312 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:20.034332 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 05:13:20.034518 (+ 186us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 05:13:20.034523 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 05:13:20.034554 (+ 31us) client_negotiation.cc:240] Negotiation successful
0811 05:13:20.034597 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260811 05:13:20.034669 11694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:20.032003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36166 (local address 127.28.143.131:40121)
0811 05:13:20.032061 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:20.032064 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:20.032072 (+ 8us) server_negotiation.cc:407] Connection header received
0811 05:13:20.032098 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:20.032099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:20.032110 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:20.032166 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:13:20.032281 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:20.033070 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:20.033870 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:20.034185 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:20.034256 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:20.034430 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:13:20.034467 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:13:20.034515 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:20.034559 (+ 44us) server_negotiation.cc:299] Negotiation successful
0811 05:13:20.034583 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 05:13:20.035770 11280 raft_consensus.cc:1261] T 6779fc4ade994e5f88a0aa56045d7f12 P 8f7748967c7344fca44f6bfe49a720de [term 1 FOLLOWER]: Refusing update from remote peer a0c7e62d8f73465585819096c63c4a53: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:20.035770 11006 raft_consensus.cc:1261] T 6779fc4ade994e5f88a0aa56045d7f12 P 9dbce36fab4e4dd695c4d4d879e301c5 [term 1 FOLLOWER]: Refusing update from remote peer a0c7e62d8f73465585819096c63c4a53: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:20.036023 11716 consensus_queue.cc:1035] T 6779fc4ade994e5f88a0aa56045d7f12 P a0c7e62d8f73465585819096c63c4a53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:20.036141 11709 consensus_queue.cc:1035] T 6779fc4ade994e5f88a0aa56045d7f12 P a0c7e62d8f73465585819096c63c4a53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:20.036585 11713 tablet_replica.cc:439] T 6779fc4ade994e5f88a0aa56045d7f12 P 9dbce36fab4e4dd695c4d4d879e301c5: TxnStatusTablet state changed. Reason: New leader a0c7e62d8f73465585819096c63c4a53. Latest consensus state: current_term: 1 leader_uuid: "a0c7e62d8f73465585819096c63c4a53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } } }
I20260811 05:13:20.036697 11713 tablet_replica.cc:442] T 6779fc4ade994e5f88a0aa56045d7f12 P 9dbce36fab4e4dd695c4d4d879e301c5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:13:20.036806 11711 tablet_replica.cc:439] T 6779fc4ade994e5f88a0aa56045d7f12 P 8f7748967c7344fca44f6bfe49a720de: TxnStatusTablet state changed. Reason: New leader a0c7e62d8f73465585819096c63c4a53. Latest consensus state: current_term: 1 leader_uuid: "a0c7e62d8f73465585819096c63c4a53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } } }
I20260811 05:13:20.036891 11711 tablet_replica.cc:442] T 6779fc4ade994e5f88a0aa56045d7f12 P 8f7748967c7344fca44f6bfe49a720de: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:13:20.037348 11716 tablet_replica.cc:439] T 6779fc4ade994e5f88a0aa56045d7f12 P a0c7e62d8f73465585819096c63c4a53: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a0c7e62d8f73465585819096c63c4a53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } } }
I20260811 05:13:20.037433 11716 tablet_replica.cc:442] T 6779fc4ade994e5f88a0aa56045d7f12 P a0c7e62d8f73465585819096c63c4a53: This TxnStatusTablet replica's current role is: LEADER
I20260811 05:13:20.037705 11711 tablet_replica.cc:439] T 6779fc4ade994e5f88a0aa56045d7f12 P 8f7748967c7344fca44f6bfe49a720de: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a0c7e62d8f73465585819096c63c4a53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } } }
I20260811 05:13:20.037781 11711 tablet_replica.cc:442] T 6779fc4ade994e5f88a0aa56045d7f12 P 8f7748967c7344fca44f6bfe49a720de: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:13:20.037798 11716 tablet_replica.cc:439] T 6779fc4ade994e5f88a0aa56045d7f12 P a0c7e62d8f73465585819096c63c4a53: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a0c7e62d8f73465585819096c63c4a53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } } }
I20260811 05:13:20.037861 11716 tablet_replica.cc:442] T 6779fc4ade994e5f88a0aa56045d7f12 P a0c7e62d8f73465585819096c63c4a53: This TxnStatusTablet replica's current role is: LEADER
I20260811 05:13:20.038185 11713 tablet_replica.cc:439] T 6779fc4ade994e5f88a0aa56045d7f12 P 9dbce36fab4e4dd695c4d4d879e301c5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a0c7e62d8f73465585819096c63c4a53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } } }
I20260811 05:13:20.038262 11713 tablet_replica.cc:442] T 6779fc4ade994e5f88a0aa56045d7f12 P 9dbce36fab4e4dd695c4d4d879e301c5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:13:20.044663 11785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:20.041754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.131:40121 (local address 127.28.143.131:59627)
0811 05:13:20.041924 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:20.041935 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:20.041989 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:20.042100 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:20.042100 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:20.042102 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 05:13:20.042156 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:20.042158 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:20.043051 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:20.043052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:20.043757 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:20.043760 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:20.044234 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:20.044235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:20.044293 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:20.044312 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 05:13:20.044473 (+ 161us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 05:13:20.044474 (+ 1us) client_negotiation.cc:769] Sending connection context
0811 05:13:20.044504 (+ 30us) client_negotiation.cc:240] Negotiation successful
0811 05:13:20.044544 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":94,"threads_started":1}
I20260811 05:13:20.044682 11694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:20.041902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:59627 (local address 127.28.143.131:40121)
0811 05:13:20.041941 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:20.041943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:20.041978 (+ 35us) server_negotiation.cc:407] Connection header received
0811 05:13:20.042028 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:20.042028 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:20.042038 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:20.042097 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:13:20.042192 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:20.042947 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:20.043842 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:20.044170 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:20.044233 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:20.044386 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:13:20.044422 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:13:20.044469 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:20.044510 (+ 41us) server_negotiation.cc:299] Negotiation successful
0811 05:13:20.044529 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":20}
I20260811 05:13:20.047245 11006 raft_consensus.cc:1261] T 0957b1f1b28b4980ac70ab15307466bc P 9dbce36fab4e4dd695c4d4d879e301c5 [term 1 FOLLOWER]: Refusing update from remote peer a0c7e62d8f73465585819096c63c4a53: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:20.047236 11280 raft_consensus.cc:1261] T 0957b1f1b28b4980ac70ab15307466bc P 8f7748967c7344fca44f6bfe49a720de [term 1 FOLLOWER]: Refusing update from remote peer a0c7e62d8f73465585819096c63c4a53: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:20.047521 11787 consensus_queue.cc:1035] T 0957b1f1b28b4980ac70ab15307466bc P a0c7e62d8f73465585819096c63c4a53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:20.047631 11709 consensus_queue.cc:1035] T 0957b1f1b28b4980ac70ab15307466bc P a0c7e62d8f73465585819096c63c4a53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:20.062142 10694 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:20.062282 10694 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:20.063007 10694 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4b9d89233cf04d7697a4dd645a3893ae: Sending DeleteTablet for 3 replicas of tablet 0957b1f1b28b4980ac70ab15307466bc
I20260811 05:13:20.063259 10967 tablet_service.cc:1514] Processing DeleteTablet for tablet 0957b1f1b28b4980ac70ab15307466bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:20 UTC) from {username='oryx', principal='oryx/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:48010
I20260811 05:13:20.063309 11246 tablet_service.cc:1514] Processing DeleteTablet for tablet 0957b1f1b28b4980ac70ab15307466bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:20 UTC) from {username='oryx', principal='oryx/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:42342
I20260811 05:13:20.063416 11741 tablet_replica.cc:330] T 0957b1f1b28b4980ac70ab15307466bc P 9dbce36fab4e4dd695c4d4d879e301c5: stopping tablet replica
I20260811 05:13:20.063450 11742 tablet_replica.cc:330] T 0957b1f1b28b4980ac70ab15307466bc P 8f7748967c7344fca44f6bfe49a720de: stopping tablet replica
I20260811 05:13:20.063405 11518 tablet_service.cc:1514] Processing DeleteTablet for tablet 0957b1f1b28b4980ac70ab15307466bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:20 UTC) from {username='oryx', principal='oryx/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:36154
I20260811 05:13:20.063473 11741 raft_consensus.cc:2229] T 0957b1f1b28b4980ac70ab15307466bc P 9dbce36fab4e4dd695c4d4d879e301c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:20.063508 11742 raft_consensus.cc:2229] T 0957b1f1b28b4980ac70ab15307466bc P 8f7748967c7344fca44f6bfe49a720de [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:20.063529 11741 raft_consensus.cc:2258] T 0957b1f1b28b4980ac70ab15307466bc P 9dbce36fab4e4dd695c4d4d879e301c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:20.063526 11740 tablet_replica.cc:330] T 0957b1f1b28b4980ac70ab15307466bc P a0c7e62d8f73465585819096c63c4a53: stopping tablet replica
I20260811 05:13:20.063549 11742 raft_consensus.cc:2258] T 0957b1f1b28b4980ac70ab15307466bc P 8f7748967c7344fca44f6bfe49a720de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:20.063581 11740 raft_consensus.cc:2229] T 0957b1f1b28b4980ac70ab15307466bc P a0c7e62d8f73465585819096c63c4a53 [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:13:20.063656 11740 raft_consensus.cc:2258] T 0957b1f1b28b4980ac70ab15307466bc P a0c7e62d8f73465585819096c63c4a53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:20.063941 11740 ts_tablet_manager.cc:1905] T 0957b1f1b28b4980ac70ab15307466bc P a0c7e62d8f73465585819096c63c4a53: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:20.064086 11742 ts_tablet_manager.cc:1905] T 0957b1f1b28b4980ac70ab15307466bc P 8f7748967c7344fca44f6bfe49a720de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:20.064137 11741 ts_tablet_manager.cc:1905] T 0957b1f1b28b4980ac70ab15307466bc P 9dbce36fab4e4dd695c4d4d879e301c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:20.065640 11740 ts_tablet_manager.cc:1918] T 0957b1f1b28b4980ac70ab15307466bc P a0c7e62d8f73465585819096c63c4a53: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 05:13:20.065656 11742 ts_tablet_manager.cc:1918] T 0957b1f1b28b4980ac70ab15307466bc P 8f7748967c7344fca44f6bfe49a720de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 05:13:20.065697 11740 log.cc:1196] T 0957b1f1b28b4980ac70ab15307466bc P a0c7e62d8f73465585819096c63c4a53: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-2/wal/wals/0957b1f1b28b4980ac70ab15307466bc
I20260811 05:13:20.065711 11742 log.cc:1196] T 0957b1f1b28b4980ac70ab15307466bc P 8f7748967c7344fca44f6bfe49a720de: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-1/wal/wals/0957b1f1b28b4980ac70ab15307466bc
I20260811 05:13:20.065912 11740 ts_tablet_manager.cc:1939] T 0957b1f1b28b4980ac70ab15307466bc P a0c7e62d8f73465585819096c63c4a53: Deleting consensus metadata
I20260811 05:13:20.065933 11742 ts_tablet_manager.cc:1939] T 0957b1f1b28b4980ac70ab15307466bc P 8f7748967c7344fca44f6bfe49a720de: Deleting consensus metadata
I20260811 05:13:20.068104 10674 catalog_manager.cc:4841] TS 8f7748967c7344fca44f6bfe49a720de (127.28.143.130:41671): tablet 0957b1f1b28b4980ac70ab15307466bc (table test-table [id=0a6b329cb65d48a59c1bd43b007f4bed]) successfully deleted
I20260811 05:13:20.068368 10674 catalog_manager.cc:4841] TS a0c7e62d8f73465585819096c63c4a53 (127.28.143.131:40121): tablet 0957b1f1b28b4980ac70ab15307466bc (table test-table [id=0a6b329cb65d48a59c1bd43b007f4bed]) successfully deleted
I20260811 05:13:20.069136 11741 ts_tablet_manager.cc:1918] T 0957b1f1b28b4980ac70ab15307466bc P 9dbce36fab4e4dd695c4d4d879e301c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 05:13:20.069198 11741 log.cc:1196] T 0957b1f1b28b4980ac70ab15307466bc P 9dbce36fab4e4dd695c4d4d879e301c5: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-0/wal/wals/0957b1f1b28b4980ac70ab15307466bc
I20260811 05:13:20.069406 11741 ts_tablet_manager.cc:1939] T 0957b1f1b28b4980ac70ab15307466bc P 9dbce36fab4e4dd695c4d4d879e301c5: Deleting consensus metadata
I20260811 05:13:20.069770 10674 catalog_manager.cc:4841] TS 9dbce36fab4e4dd695c4d4d879e301c5 (127.28.143.129:37049): tablet 0957b1f1b28b4980ac70ab15307466bc (table test-table [id=0a6b329cb65d48a59c1bd43b007f4bed]) successfully deleted
Aug 11 05:13:20 dist-test-slave-f7th krb5kdc[10615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425200, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 05:13:20 dist-test-slave-f7th krb5kdc[10615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:20 dist-test-slave-f7th krb5kdc[10615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 05:13:20.091432 10790 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:20.085287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57574 (local address 127.28.143.190:39863)
0811 05:13:20.085355 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:20.085357 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:20.085461 (+ 104us) server_negotiation.cc:407] Connection header received
0811 05:13:20.085543 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:20.085544 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:20.085556 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:20.085606 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:20.086342 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:20.086920 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:20.087990 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:20.088187 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:20.088244 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:20.091351 (+ 3107us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57574: BlockingRecv error: recv got EOF from 127.0.0.1:57574 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 11 05:13:20 dist-test-slave-f7th krb5kdc[10615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425200, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM
I20260811 05:13:20.100894 10790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:20.092370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57586 (local address 127.28.143.190:39863)
0811 05:13:20.092419 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:20.092421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:20.092432 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:13:20.092484 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:20.092484 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:20.092495 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:20.092568 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:20.093177 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:20.094050 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:20.094764 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:20.095038 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:20.095112 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:20.097726 (+ 2614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:20.097728 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:20.097728 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:20.097746 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:20.099613 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:20.100071 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:20.100072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:20.100073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:20.100130 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:20.100339 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:20.100339 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:20.100340 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:20.100460 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:20.100535 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:20.100735 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 05:13:20.100799 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 05:13:20.102447 10694 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57586:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:13:20.102598 10694 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:13:20.104010 10967 tablet_service.cc:1467] Processing CreateTablet for tablet 8d0d86194af343249c02cf1ec0f98571 (DEFAULT_TABLE table=test-table [id=1e5279ead2744389a15ec4cc5deaf897]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:20.104035 11518 tablet_service.cc:1467] Processing CreateTablet for tablet 8d0d86194af343249c02cf1ec0f98571 (DEFAULT_TABLE table=test-table [id=1e5279ead2744389a15ec4cc5deaf897]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:20.104167 11518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d0d86194af343249c02cf1ec0f98571. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:20.104169 10967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d0d86194af343249c02cf1ec0f98571. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:20.105330 11246 tablet_service.cc:1467] Processing CreateTablet for tablet 8d0d86194af343249c02cf1ec0f98571 (DEFAULT_TABLE table=test-table [id=1e5279ead2744389a15ec4cc5deaf897]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:20.105465 11246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d0d86194af343249c02cf1ec0f98571. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:20.106452 11703 tablet_bootstrap.cc:492] T 8d0d86194af343249c02cf1ec0f98571 P 8f7748967c7344fca44f6bfe49a720de: Bootstrap starting.
I20260811 05:13:20.106467 11702 tablet_bootstrap.cc:492] T 8d0d86194af343249c02cf1ec0f98571 P a0c7e62d8f73465585819096c63c4a53: Bootstrap starting.
I20260811 05:13:20.106777 11702 tablet_bootstrap.cc:654] T 8d0d86194af343249c02cf1ec0f98571 P a0c7e62d8f73465585819096c63c4a53: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:20.106809 11703 tablet_bootstrap.cc:654] T 8d0d86194af343249c02cf1ec0f98571 P 8f7748967c7344fca44f6bfe49a720de: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:20.107110 11706 tablet_bootstrap.cc:492] T 8d0d86194af343249c02cf1ec0f98571 P 9dbce36fab4e4dd695c4d4d879e301c5: Bootstrap starting.
I20260811 05:13:20.107520 11706 tablet_bootstrap.cc:654] T 8d0d86194af343249c02cf1ec0f98571 P 9dbce36fab4e4dd695c4d4d879e301c5: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:20.108048 11702 tablet_bootstrap.cc:492] T 8d0d86194af343249c02cf1ec0f98571 P a0c7e62d8f73465585819096c63c4a53: No bootstrap required, opened a new log
I20260811 05:13:20.108103 11702 ts_tablet_manager.cc:1397] T 8d0d86194af343249c02cf1ec0f98571 P a0c7e62d8f73465585819096c63c4a53: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 05:13:20.108202 11706 tablet_bootstrap.cc:492] T 8d0d86194af343249c02cf1ec0f98571 P 9dbce36fab4e4dd695c4d4d879e301c5: No bootstrap required, opened a new log
I20260811 05:13:20.108218 11702 raft_consensus.cc:348] T 8d0d86194af343249c02cf1ec0f98571 P a0c7e62d8f73465585819096c63c4a53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } }
I20260811 05:13:20.108263 11706 ts_tablet_manager.cc:1397] T 8d0d86194af343249c02cf1ec0f98571 P 9dbce36fab4e4dd695c4d4d879e301c5: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:20.108289 11702 raft_consensus.cc:374] T 8d0d86194af343249c02cf1ec0f98571 P a0c7e62d8f73465585819096c63c4a53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:20.108309 11702 raft_consensus.cc:729] T 8d0d86194af343249c02cf1ec0f98571 P a0c7e62d8f73465585819096c63c4a53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0c7e62d8f73465585819096c63c4a53, State: Initialized, Role: FOLLOWER
I20260811 05:13:20.108348 11702 consensus_queue.cc:260] T 8d0d86194af343249c02cf1ec0f98571 P a0c7e62d8f73465585819096c63c4a53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } }
I20260811 05:13:20.108431 11702 ts_tablet_manager.cc:1428] T 8d0d86194af343249c02cf1ec0f98571 P a0c7e62d8f73465585819096c63c4a53: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:20.108417 11706 raft_consensus.cc:348] T 8d0d86194af343249c02cf1ec0f98571 P 9dbce36fab4e4dd695c4d4d879e301c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } }
I20260811 05:13:20.108486 11706 raft_consensus.cc:374] T 8d0d86194af343249c02cf1ec0f98571 P 9dbce36fab4e4dd695c4d4d879e301c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:20.108507 11706 raft_consensus.cc:729] T 8d0d86194af343249c02cf1ec0f98571 P 9dbce36fab4e4dd695c4d4d879e301c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9dbce36fab4e4dd695c4d4d879e301c5, State: Initialized, Role: FOLLOWER
I20260811 05:13:20.108551 11706 consensus_queue.cc:260] T 8d0d86194af343249c02cf1ec0f98571 P 9dbce36fab4e4dd695c4d4d879e301c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } }
I20260811 05:13:20.108697 11706 ts_tablet_manager.cc:1428] T 8d0d86194af343249c02cf1ec0f98571 P 9dbce36fab4e4dd695c4d4d879e301c5: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 05:13:20.108903 11703 tablet_bootstrap.cc:492] T 8d0d86194af343249c02cf1ec0f98571 P 8f7748967c7344fca44f6bfe49a720de: No bootstrap required, opened a new log
I20260811 05:13:20.108950 11703 ts_tablet_manager.cc:1397] T 8d0d86194af343249c02cf1ec0f98571 P 8f7748967c7344fca44f6bfe49a720de: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260811 05:13:20.109086 11703 raft_consensus.cc:348] T 8d0d86194af343249c02cf1ec0f98571 P 8f7748967c7344fca44f6bfe49a720de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } }
I20260811 05:13:20.109136 11703 raft_consensus.cc:374] T 8d0d86194af343249c02cf1ec0f98571 P 8f7748967c7344fca44f6bfe49a720de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:20.109148 11703 raft_consensus.cc:729] T 8d0d86194af343249c02cf1ec0f98571 P 8f7748967c7344fca44f6bfe49a720de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f7748967c7344fca44f6bfe49a720de, State: Initialized, Role: FOLLOWER
I20260811 05:13:20.109184 11703 consensus_queue.cc:260] T 8d0d86194af343249c02cf1ec0f98571 P 8f7748967c7344fca44f6bfe49a720de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } }
I20260811 05:13:20.109263 11703 ts_tablet_manager.cc:1428] T 8d0d86194af343249c02cf1ec0f98571 P 8f7748967c7344fca44f6bfe49a720de: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:20.183038 11716 raft_consensus.cc:482] T 8d0d86194af343249c02cf1ec0f98571 P a0c7e62d8f73465585819096c63c4a53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:13:20.183154 11716 raft_consensus.cc:504] T 8d0d86194af343249c02cf1ec0f98571 P a0c7e62d8f73465585819096c63c4a53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } }
I20260811 05:13:20.183290 11716 leader_election.cc:288] T 8d0d86194af343249c02cf1ec0f98571 P a0c7e62d8f73465585819096c63c4a53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9dbce36fab4e4dd695c4d4d879e301c5 (127.28.143.129:37049), 8f7748967c7344fca44f6bfe49a720de (127.28.143.130:41671)
I20260811 05:13:20.183694 11006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d0d86194af343249c02cf1ec0f98571" candidate_uuid: "a0c7e62d8f73465585819096c63c4a53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" is_pre_election: true
I20260811 05:13:20.183696 11280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d0d86194af343249c02cf1ec0f98571" candidate_uuid: "a0c7e62d8f73465585819096c63c4a53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f7748967c7344fca44f6bfe49a720de" is_pre_election: true
I20260811 05:13:20.183816 11006 raft_consensus.cc:2454] T 8d0d86194af343249c02cf1ec0f98571 P 9dbce36fab4e4dd695c4d4d879e301c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0c7e62d8f73465585819096c63c4a53 in term 0.
I20260811 05:13:20.183816 11280 raft_consensus.cc:2454] T 8d0d86194af343249c02cf1ec0f98571 P 8f7748967c7344fca44f6bfe49a720de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0c7e62d8f73465585819096c63c4a53 in term 0.
I20260811 05:13:20.184041 11426 leader_election.cc:302] T 8d0d86194af343249c02cf1ec0f98571 P a0c7e62d8f73465585819096c63c4a53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9dbce36fab4e4dd695c4d4d879e301c5, a0c7e62d8f73465585819096c63c4a53; no voters:
I20260811 05:13:20.184168 11716 raft_consensus.cc:2781] T 8d0d86194af343249c02cf1ec0f98571 P a0c7e62d8f73465585819096c63c4a53 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:13:20.184234 11716 raft_consensus.cc:482] T 8d0d86194af343249c02cf1ec0f98571 P a0c7e62d8f73465585819096c63c4a53 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:13:20.184249 11716 raft_consensus.cc:3037] T 8d0d86194af343249c02cf1ec0f98571 P a0c7e62d8f73465585819096c63c4a53 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:20.185072 11716 raft_consensus.cc:504] T 8d0d86194af343249c02cf1ec0f98571 P a0c7e62d8f73465585819096c63c4a53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } }
I20260811 05:13:20.185209 11716 leader_election.cc:288] T 8d0d86194af343249c02cf1ec0f98571 P a0c7e62d8f73465585819096c63c4a53 [CANDIDATE]: Term 1 election: Requested vote from peers 9dbce36fab4e4dd695c4d4d879e301c5 (127.28.143.129:37049), 8f7748967c7344fca44f6bfe49a720de (127.28.143.130:41671)
I20260811 05:13:20.185460 11006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d0d86194af343249c02cf1ec0f98571" candidate_uuid: "a0c7e62d8f73465585819096c63c4a53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9dbce36fab4e4dd695c4d4d879e301c5"
I20260811 05:13:20.185460 11280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d0d86194af343249c02cf1ec0f98571" candidate_uuid: "a0c7e62d8f73465585819096c63c4a53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f7748967c7344fca44f6bfe49a720de"
I20260811 05:13:20.185561 11006 raft_consensus.cc:3037] T 8d0d86194af343249c02cf1ec0f98571 P 9dbce36fab4e4dd695c4d4d879e301c5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:20.185576 11280 raft_consensus.cc:3037] T 8d0d86194af343249c02cf1ec0f98571 P 8f7748967c7344fca44f6bfe49a720de [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:20.186362 11280 raft_consensus.cc:2454] T 8d0d86194af343249c02cf1ec0f98571 P 8f7748967c7344fca44f6bfe49a720de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0c7e62d8f73465585819096c63c4a53 in term 1.
I20260811 05:13:20.186362 11006 raft_consensus.cc:2454] T 8d0d86194af343249c02cf1ec0f98571 P 9dbce36fab4e4dd695c4d4d879e301c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0c7e62d8f73465585819096c63c4a53 in term 1.
I20260811 05:13:20.186586 11426 leader_election.cc:302] T 8d0d86194af343249c02cf1ec0f98571 P a0c7e62d8f73465585819096c63c4a53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9dbce36fab4e4dd695c4d4d879e301c5, a0c7e62d8f73465585819096c63c4a53; no voters:
I20260811 05:13:20.186703 11716 raft_consensus.cc:2781] T 8d0d86194af343249c02cf1ec0f98571 P a0c7e62d8f73465585819096c63c4a53 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:20.186762 11716 raft_consensus.cc:686] T 8d0d86194af343249c02cf1ec0f98571 P a0c7e62d8f73465585819096c63c4a53 [term 1 LEADER]: Becoming Leader. State: Replica: a0c7e62d8f73465585819096c63c4a53, State: Running, Role: LEADER
I20260811 05:13:20.186827 11716 consensus_queue.cc:237] T 8d0d86194af343249c02cf1ec0f98571 P a0c7e62d8f73465585819096c63c4a53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } }
I20260811 05:13:20.187448 10695 catalog_manager.cc:5495] T 8d0d86194af343249c02cf1ec0f98571 P a0c7e62d8f73465585819096c63c4a53 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0c7e62d8f73465585819096c63c4a53 (127.28.143.131). New cstate: current_term: 1 leader_uuid: "a0c7e62d8f73465585819096c63c4a53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } health_report { overall_health: HEALTHY } } }
I20260811 05:13:20.203039 11694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:20.200926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36176 (local address 127.28.143.131:40121)
0811 05:13:20.200981 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:20.200983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:20.200994 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:13:20.201051 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:20.201052 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:20.201062 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:20.201113 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:13:20.201233 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:20.201714 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:20.202419 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:20.202619 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:20.202675 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:20.202804 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:13:20.202838 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:13:20.202878 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:20.202930 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 05:13:20.202951 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 05:13:20.203797 11280 raft_consensus.cc:1261] T 8d0d86194af343249c02cf1ec0f98571 P 8f7748967c7344fca44f6bfe49a720de [term 1 FOLLOWER]: Refusing update from remote peer a0c7e62d8f73465585819096c63c4a53: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:20.203850 11006 raft_consensus.cc:1261] T 8d0d86194af343249c02cf1ec0f98571 P 9dbce36fab4e4dd695c4d4d879e301c5 [term 1 FOLLOWER]: Refusing update from remote peer a0c7e62d8f73465585819096c63c4a53: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:20.204066 11787 consensus_queue.cc:1035] T 8d0d86194af343249c02cf1ec0f98571 P a0c7e62d8f73465585819096c63c4a53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:20.204195 11716 consensus_queue.cc:1035] T 8d0d86194af343249c02cf1ec0f98571 P a0c7e62d8f73465585819096c63c4a53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:20.207424 10695 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:20.207530 10695 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:20.208166 10695 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4b9d89233cf04d7697a4dd645a3893ae: Sending DeleteTablet for 3 replicas of tablet 8d0d86194af343249c02cf1ec0f98571
I20260811 05:13:20.208413 10967 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d0d86194af343249c02cf1ec0f98571 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:20 UTC) from {username='oryx', principal='oryx/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:48010
I20260811 05:13:20.208508 11518 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d0d86194af343249c02cf1ec0f98571 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:20 UTC) from {username='oryx', principal='oryx/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:36154
I20260811 05:13:20.208508 11246 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d0d86194af343249c02cf1ec0f98571 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:20 UTC) from {username='oryx', principal='oryx/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:42342
I20260811 05:13:20.208540 11741 tablet_replica.cc:330] T 8d0d86194af343249c02cf1ec0f98571 P 9dbce36fab4e4dd695c4d4d879e301c5: stopping tablet replica
I20260811 05:13:20.208602 11741 raft_consensus.cc:2229] T 8d0d86194af343249c02cf1ec0f98571 P 9dbce36fab4e4dd695c4d4d879e301c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:20.208634 11740 tablet_replica.cc:330] T 8d0d86194af343249c02cf1ec0f98571 P a0c7e62d8f73465585819096c63c4a53: stopping tablet replica
I20260811 05:13:20.208693 11741 raft_consensus.cc:2258] T 8d0d86194af343249c02cf1ec0f98571 P 9dbce36fab4e4dd695c4d4d879e301c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:20.208693 11740 raft_consensus.cc:2229] T 8d0d86194af343249c02cf1ec0f98571 P a0c7e62d8f73465585819096c63c4a53 [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:13:20.208679 11742 tablet_replica.cc:330] T 8d0d86194af343249c02cf1ec0f98571 P 8f7748967c7344fca44f6bfe49a720de: stopping tablet replica
I20260811 05:13:20.208753 11742 raft_consensus.cc:2229] T 8d0d86194af343249c02cf1ec0f98571 P 8f7748967c7344fca44f6bfe49a720de [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:20.208757 11740 raft_consensus.cc:2258] T 8d0d86194af343249c02cf1ec0f98571 P a0c7e62d8f73465585819096c63c4a53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:20.208808 11742 raft_consensus.cc:2258] T 8d0d86194af343249c02cf1ec0f98571 P 8f7748967c7344fca44f6bfe49a720de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:20.209071 11741 ts_tablet_manager.cc:1905] T 8d0d86194af343249c02cf1ec0f98571 P 9dbce36fab4e4dd695c4d4d879e301c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:20.209079 11740 ts_tablet_manager.cc:1905] T 8d0d86194af343249c02cf1ec0f98571 P a0c7e62d8f73465585819096c63c4a53: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:20.209209 11742 ts_tablet_manager.cc:1905] T 8d0d86194af343249c02cf1ec0f98571 P 8f7748967c7344fca44f6bfe49a720de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:20.210623 11741 ts_tablet_manager.cc:1918] T 8d0d86194af343249c02cf1ec0f98571 P 9dbce36fab4e4dd695c4d4d879e301c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:13:20.210652 11742 ts_tablet_manager.cc:1918] T 8d0d86194af343249c02cf1ec0f98571 P 8f7748967c7344fca44f6bfe49a720de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:13:20.210690 11741 log.cc:1196] T 8d0d86194af343249c02cf1ec0f98571 P 9dbce36fab4e4dd695c4d4d879e301c5: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-0/wal/wals/8d0d86194af343249c02cf1ec0f98571
I20260811 05:13:20.210697 11742 log.cc:1196] T 8d0d86194af343249c02cf1ec0f98571 P 8f7748967c7344fca44f6bfe49a720de: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-1/wal/wals/8d0d86194af343249c02cf1ec0f98571
I20260811 05:13:20.210711 11740 ts_tablet_manager.cc:1918] T 8d0d86194af343249c02cf1ec0f98571 P a0c7e62d8f73465585819096c63c4a53: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:13:20.210762 11740 log.cc:1196] T 8d0d86194af343249c02cf1ec0f98571 P a0c7e62d8f73465585819096c63c4a53: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-2/wal/wals/8d0d86194af343249c02cf1ec0f98571
I20260811 05:13:20.210695 10695 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57586:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:13:20.210871 10695 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:13:20.210899 11742 ts_tablet_manager.cc:1939] T 8d0d86194af343249c02cf1ec0f98571 P 8f7748967c7344fca44f6bfe49a720de: Deleting consensus metadata
I20260811 05:13:20.210902 11741 ts_tablet_manager.cc:1939] T 8d0d86194af343249c02cf1ec0f98571 P 9dbce36fab4e4dd695c4d4d879e301c5: Deleting consensus metadata
I20260811 05:13:20.210980 11740 ts_tablet_manager.cc:1939] T 8d0d86194af343249c02cf1ec0f98571 P a0c7e62d8f73465585819096c63c4a53: Deleting consensus metadata
I20260811 05:13:20.211416 10674 catalog_manager.cc:4841] TS a0c7e62d8f73465585819096c63c4a53 (127.28.143.131:40121): tablet 8d0d86194af343249c02cf1ec0f98571 (table test-table [id=1e5279ead2744389a15ec4cc5deaf897]) successfully deleted
I20260811 05:13:20.211498 10674 catalog_manager.cc:4841] TS 9dbce36fab4e4dd695c4d4d879e301c5 (127.28.143.129:37049): tablet 8d0d86194af343249c02cf1ec0f98571 (table test-table [id=1e5279ead2744389a15ec4cc5deaf897]) successfully deleted
I20260811 05:13:20.211788 10674 catalog_manager.cc:4841] TS 8f7748967c7344fca44f6bfe49a720de (127.28.143.130:41671): tablet 8d0d86194af343249c02cf1ec0f98571 (table test-table [id=1e5279ead2744389a15ec4cc5deaf897]) successfully deleted
I20260811 05:13:20.212188 11246 tablet_service.cc:1467] Processing CreateTablet for tablet 21c4bdf7394e405aabc2a436c49ef73f (DEFAULT_TABLE table=test-table [id=50cad0fe362149a7aa2dd9ccfe47188c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:20.212221 11518 tablet_service.cc:1467] Processing CreateTablet for tablet 21c4bdf7394e405aabc2a436c49ef73f (DEFAULT_TABLE table=test-table [id=50cad0fe362149a7aa2dd9ccfe47188c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:20.212306 11246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21c4bdf7394e405aabc2a436c49ef73f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:20.212320 11518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21c4bdf7394e405aabc2a436c49ef73f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:20.213263 11702 tablet_bootstrap.cc:492] T 21c4bdf7394e405aabc2a436c49ef73f P a0c7e62d8f73465585819096c63c4a53: Bootstrap starting.
I20260811 05:13:20.213311 11703 tablet_bootstrap.cc:492] T 21c4bdf7394e405aabc2a436c49ef73f P 8f7748967c7344fca44f6bfe49a720de: Bootstrap starting.
I20260811 05:13:20.213356 10967 tablet_service.cc:1467] Processing CreateTablet for tablet 21c4bdf7394e405aabc2a436c49ef73f (DEFAULT_TABLE table=test-table [id=50cad0fe362149a7aa2dd9ccfe47188c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:20.213474 10967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21c4bdf7394e405aabc2a436c49ef73f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:20.213605 11702 tablet_bootstrap.cc:654] T 21c4bdf7394e405aabc2a436c49ef73f P a0c7e62d8f73465585819096c63c4a53: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:20.213657 11703 tablet_bootstrap.cc:654] T 21c4bdf7394e405aabc2a436c49ef73f P 8f7748967c7344fca44f6bfe49a720de: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:20.214514 11703 tablet_bootstrap.cc:492] T 21c4bdf7394e405aabc2a436c49ef73f P 8f7748967c7344fca44f6bfe49a720de: No bootstrap required, opened a new log
I20260811 05:13:20.214557 11703 ts_tablet_manager.cc:1397] T 21c4bdf7394e405aabc2a436c49ef73f P 8f7748967c7344fca44f6bfe49a720de: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:20.214644 11706 tablet_bootstrap.cc:492] T 21c4bdf7394e405aabc2a436c49ef73f P 9dbce36fab4e4dd695c4d4d879e301c5: Bootstrap starting.
I20260811 05:13:20.214690 11703 raft_consensus.cc:348] T 21c4bdf7394e405aabc2a436c49ef73f P 8f7748967c7344fca44f6bfe49a720de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } }
I20260811 05:13:20.214766 11703 raft_consensus.cc:374] T 21c4bdf7394e405aabc2a436c49ef73f P 8f7748967c7344fca44f6bfe49a720de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:20.214797 11703 raft_consensus.cc:729] T 21c4bdf7394e405aabc2a436c49ef73f P 8f7748967c7344fca44f6bfe49a720de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f7748967c7344fca44f6bfe49a720de, State: Initialized, Role: FOLLOWER
I20260811 05:13:20.214848 11703 consensus_queue.cc:260] T 21c4bdf7394e405aabc2a436c49ef73f P 8f7748967c7344fca44f6bfe49a720de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } }
I20260811 05:13:20.214941 11703 ts_tablet_manager.cc:1428] T 21c4bdf7394e405aabc2a436c49ef73f P 8f7748967c7344fca44f6bfe49a720de: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 05:13:20.215005 11702 tablet_bootstrap.cc:492] T 21c4bdf7394e405aabc2a436c49ef73f P a0c7e62d8f73465585819096c63c4a53: No bootstrap required, opened a new log
I20260811 05:13:20.215058 11702 ts_tablet_manager.cc:1397] T 21c4bdf7394e405aabc2a436c49ef73f P a0c7e62d8f73465585819096c63c4a53: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 05:13:20.215168 11702 raft_consensus.cc:348] T 21c4bdf7394e405aabc2a436c49ef73f P a0c7e62d8f73465585819096c63c4a53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } }
I20260811 05:13:20.215251 11702 raft_consensus.cc:374] T 21c4bdf7394e405aabc2a436c49ef73f P a0c7e62d8f73465585819096c63c4a53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:20.215270 11702 raft_consensus.cc:729] T 21c4bdf7394e405aabc2a436c49ef73f P a0c7e62d8f73465585819096c63c4a53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0c7e62d8f73465585819096c63c4a53, State: Initialized, Role: FOLLOWER
I20260811 05:13:20.215306 11706 tablet_bootstrap.cc:654] T 21c4bdf7394e405aabc2a436c49ef73f P 9dbce36fab4e4dd695c4d4d879e301c5: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:20.215317 11702 consensus_queue.cc:260] T 21c4bdf7394e405aabc2a436c49ef73f P a0c7e62d8f73465585819096c63c4a53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } }
I20260811 05:13:20.215418 11702 ts_tablet_manager.cc:1428] T 21c4bdf7394e405aabc2a436c49ef73f P a0c7e62d8f73465585819096c63c4a53: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:20.216063 11706 tablet_bootstrap.cc:492] T 21c4bdf7394e405aabc2a436c49ef73f P 9dbce36fab4e4dd695c4d4d879e301c5: No bootstrap required, opened a new log
I20260811 05:13:20.216116 11706 ts_tablet_manager.cc:1397] T 21c4bdf7394e405aabc2a436c49ef73f P 9dbce36fab4e4dd695c4d4d879e301c5: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260811 05:13:20.216252 11706 raft_consensus.cc:348] T 21c4bdf7394e405aabc2a436c49ef73f P 9dbce36fab4e4dd695c4d4d879e301c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } }
I20260811 05:13:20.216327 11706 raft_consensus.cc:374] T 21c4bdf7394e405aabc2a436c49ef73f P 9dbce36fab4e4dd695c4d4d879e301c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:20.216351 11706 raft_consensus.cc:729] T 21c4bdf7394e405aabc2a436c49ef73f P 9dbce36fab4e4dd695c4d4d879e301c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9dbce36fab4e4dd695c4d4d879e301c5, State: Initialized, Role: FOLLOWER
I20260811 05:13:20.216398 11706 consensus_queue.cc:260] T 21c4bdf7394e405aabc2a436c49ef73f P 9dbce36fab4e4dd695c4d4d879e301c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } }
I20260811 05:13:20.216488 11706 ts_tablet_manager.cc:1428] T 21c4bdf7394e405aabc2a436c49ef73f P 9dbce36fab4e4dd695c4d4d879e301c5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:20.442811 11711 raft_consensus.cc:482] T 21c4bdf7394e405aabc2a436c49ef73f P 8f7748967c7344fca44f6bfe49a720de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:13:20.442916 11711 raft_consensus.cc:504] T 21c4bdf7394e405aabc2a436c49ef73f P 8f7748967c7344fca44f6bfe49a720de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } }
I20260811 05:13:20.443243 11711 leader_election.cc:288] T 21c4bdf7394e405aabc2a436c49ef73f P 8f7748967c7344fca44f6bfe49a720de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9dbce36fab4e4dd695c4d4d879e301c5 (127.28.143.129:37049), a0c7e62d8f73465585819096c63c4a53 (127.28.143.131:40121)
I20260811 05:13:20.446273 11694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:20.443801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:56461 (local address 127.28.143.131:40121)
0811 05:13:20.443863 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:20.443866 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:20.443873 (+ 7us) server_negotiation.cc:407] Connection header received
0811 05:13:20.443900 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:20.443901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:20.443914 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:20.443970 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:20.444138 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:20.444591 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:20.445596 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:20.445972 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:20.446038 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:20.446048 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:20.446175 (+ 127us) server_negotiation.cc:299] Negotiation successful
0811 05:13:20.446209 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 05:13:20.446324 11847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:20.443643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.131:40121 (local address 127.28.143.130:56461)
0811 05:13:20.443739 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:20.443777 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:20.443827 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:20.443988 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:20.443989 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:20.443991 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:20.444063 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:20.444065 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:20.444695 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:20.444696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:20.445509 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:20.445511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:20.446049 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:20.446050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:20.446112 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:20.446116 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:13:20.446161 (+ 45us) client_negotiation.cc:240] Negotiation successful
0811 05:13:20.446200 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"mutex_wait_us":1,"thread_start_us":56,"threads_started":1}
I20260811 05:13:20.446346 11839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:20.443466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:37049 (local address 127.28.143.130:60699)
0811 05:13:20.443595 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:20.443606 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:20.443659 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:20.443908 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:20.443909 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:20.443912 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:20.443997 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:20.443999 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:20.444703 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:20.444703 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:20.445506 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:20.445508 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:20.446081 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:20.446082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:20.446143 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:20.446148 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 05:13:20.446192 (+ 44us) client_negotiation.cc:240] Negotiation successful
0811 05:13:20.446232 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"mutex_wait_us":46,"thread_start_us":66,"threads_started":1}
I20260811 05:13:20.446328 11841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:20.443548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:60699 (local address 127.28.143.129:37049)
0811 05:13:20.443662 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:20.443664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:20.443672 (+ 8us) server_negotiation.cc:407] Connection header received
0811 05:13:20.443751 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:20.443752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:20.443764 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:20.443845 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:20.444080 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:20.444608 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:20.445597 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:20.446001 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:20.446080 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:20.446089 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:20.446206 (+ 117us) server_negotiation.cc:299] Negotiation successful
0811 05:13:20.446246 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":58,"threads_started":1}
I20260811 05:13:20.446592 11006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21c4bdf7394e405aabc2a436c49ef73f" candidate_uuid: "8f7748967c7344fca44f6bfe49a720de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" is_pre_election: true
I20260811 05:13:20.446681 11006 raft_consensus.cc:2454] T 21c4bdf7394e405aabc2a436c49ef73f P 9dbce36fab4e4dd695c4d4d879e301c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f7748967c7344fca44f6bfe49a720de in term 0.
I20260811 05:13:20.446733 11555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21c4bdf7394e405aabc2a436c49ef73f" candidate_uuid: "8f7748967c7344fca44f6bfe49a720de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0c7e62d8f73465585819096c63c4a53" is_pre_election: true
I20260811 05:13:20.446894 11555 raft_consensus.cc:2454] T 21c4bdf7394e405aabc2a436c49ef73f P a0c7e62d8f73465585819096c63c4a53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f7748967c7344fca44f6bfe49a720de in term 0.
I20260811 05:13:20.446889 11147 leader_election.cc:302] T 21c4bdf7394e405aabc2a436c49ef73f P 8f7748967c7344fca44f6bfe49a720de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f7748967c7344fca44f6bfe49a720de, 9dbce36fab4e4dd695c4d4d879e301c5; no voters:
I20260811 05:13:20.447019 11711 raft_consensus.cc:2781] T 21c4bdf7394e405aabc2a436c49ef73f P 8f7748967c7344fca44f6bfe49a720de [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:13:20.447063 11711 raft_consensus.cc:482] T 21c4bdf7394e405aabc2a436c49ef73f P 8f7748967c7344fca44f6bfe49a720de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:13:20.447090 11711 raft_consensus.cc:3037] T 21c4bdf7394e405aabc2a436c49ef73f P 8f7748967c7344fca44f6bfe49a720de [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:20.447836 11711 raft_consensus.cc:504] T 21c4bdf7394e405aabc2a436c49ef73f P 8f7748967c7344fca44f6bfe49a720de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } }
I20260811 05:13:20.447976 11711 leader_election.cc:288] T 21c4bdf7394e405aabc2a436c49ef73f P 8f7748967c7344fca44f6bfe49a720de [CANDIDATE]: Term 1 election: Requested vote from peers 9dbce36fab4e4dd695c4d4d879e301c5 (127.28.143.129:37049), a0c7e62d8f73465585819096c63c4a53 (127.28.143.131:40121)
I20260811 05:13:20.449080 11555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21c4bdf7394e405aabc2a436c49ef73f" candidate_uuid: "8f7748967c7344fca44f6bfe49a720de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0c7e62d8f73465585819096c63c4a53"
I20260811 05:13:20.449080 11006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21c4bdf7394e405aabc2a436c49ef73f" candidate_uuid: "8f7748967c7344fca44f6bfe49a720de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9dbce36fab4e4dd695c4d4d879e301c5"
I20260811 05:13:20.449170 11555 raft_consensus.cc:3037] T 21c4bdf7394e405aabc2a436c49ef73f P a0c7e62d8f73465585819096c63c4a53 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:20.449170 11006 raft_consensus.cc:3037] T 21c4bdf7394e405aabc2a436c49ef73f P 9dbce36fab4e4dd695c4d4d879e301c5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:20.449973 11006 raft_consensus.cc:2454] T 21c4bdf7394e405aabc2a436c49ef73f P 9dbce36fab4e4dd695c4d4d879e301c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f7748967c7344fca44f6bfe49a720de in term 1.
I20260811 05:13:20.449973 11555 raft_consensus.cc:2454] T 21c4bdf7394e405aabc2a436c49ef73f P a0c7e62d8f73465585819096c63c4a53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f7748967c7344fca44f6bfe49a720de in term 1.
I20260811 05:13:20.450179 11147 leader_election.cc:302] T 21c4bdf7394e405aabc2a436c49ef73f P 8f7748967c7344fca44f6bfe49a720de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f7748967c7344fca44f6bfe49a720de, a0c7e62d8f73465585819096c63c4a53; no voters:
I20260811 05:13:20.450274 11711 raft_consensus.cc:2781] T 21c4bdf7394e405aabc2a436c49ef73f P 8f7748967c7344fca44f6bfe49a720de [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:20.450417 11711 raft_consensus.cc:686] T 21c4bdf7394e405aabc2a436c49ef73f P 8f7748967c7344fca44f6bfe49a720de [term 1 LEADER]: Becoming Leader. State: Replica: 8f7748967c7344fca44f6bfe49a720de, State: Running, Role: LEADER
I20260811 05:13:20.450503 11711 consensus_queue.cc:237] T 21c4bdf7394e405aabc2a436c49ef73f P 8f7748967c7344fca44f6bfe49a720de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } }
I20260811 05:13:20.451295 10694 catalog_manager.cc:5495] T 21c4bdf7394e405aabc2a436c49ef73f P 8f7748967c7344fca44f6bfe49a720de reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f7748967c7344fca44f6bfe49a720de (127.28.143.130). New cstate: current_term: 1 leader_uuid: "8f7748967c7344fca44f6bfe49a720de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f7748967c7344fca44f6bfe49a720de" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41671 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 } health_report { overall_health: UNKNOWN } } }
I20260811 05:13:20.505395 11851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:20.502794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42348 (local address 127.28.143.130:41671)
0811 05:13:20.502954 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:20.502956 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:20.502962 (+ 6us) server_negotiation.cc:407] Connection header received
0811 05:13:20.503001 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:20.503002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:20.503024 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:20.503100 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:13:20.503248 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:20.503808 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:20.504517 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:20.504795 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:20.504851 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:20.505022 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:13:20.505150 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:13:20.505219 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:20.505256 (+ 37us) server_negotiation.cc:299] Negotiation successful
0811 05:13:20.505278 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":96,"threads_started":1}
I20260811 05:13:20.510715 11856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:20.508203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.131:40121 (local address 127.28.143.130:53039)
0811 05:13:20.508372 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:20.508384 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:20.508443 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:20.508554 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:20.508554 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:20.508556 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 05:13:20.508646 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:20.508649 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:20.509256 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:20.509257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:20.509875 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:20.509877 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:20.510252 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:20.510252 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:20.510320 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:20.510338 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 05:13:20.510508 (+ 170us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 05:13:20.510510 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 05:13:20.510558 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 05:13:20.510624 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":74,"threads_started":1}
I20260811 05:13:20.510699 11694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:20.508323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:53039 (local address 127.28.143.131:40121)
0811 05:13:20.508376 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:20.508379 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:20.508455 (+ 76us) server_negotiation.cc:407] Connection header received
0811 05:13:20.508488 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:20.508489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:20.508499 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:20.508547 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:13:20.508706 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:20.509181 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:20.509958 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:20.510188 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:20.510244 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:20.510420 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:13:20.510458 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:13:20.510502 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:20.510562 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 05:13:20.510583 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 05:13:20.512977 11006 raft_consensus.cc:1261] T 21c4bdf7394e405aabc2a436c49ef73f P 9dbce36fab4e4dd695c4d4d879e301c5 [term 1 FOLLOWER]: Refusing update from remote peer 8f7748967c7344fca44f6bfe49a720de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:20.513019 11555 raft_consensus.cc:1261] T 21c4bdf7394e405aabc2a436c49ef73f P a0c7e62d8f73465585819096c63c4a53 [term 1 FOLLOWER]: Refusing update from remote peer 8f7748967c7344fca44f6bfe49a720de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:20.513283 11849 consensus_queue.cc:1035] T 21c4bdf7394e405aabc2a436c49ef73f P 8f7748967c7344fca44f6bfe49a720de [LEADER]: Connected to new peer: Peer: permanent_uuid: "9dbce36fab4e4dd695c4d4d879e301c5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:20.513419 11711 consensus_queue.cc:1035] T 21c4bdf7394e405aabc2a436c49ef73f P 8f7748967c7344fca44f6bfe49a720de [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0c7e62d8f73465585819096c63c4a53" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 40121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:20.520505 11851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:20.518152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:33179 (local address 127.28.143.130:41671)
0811 05:13:20.518286 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:20.518288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:20.518293 (+ 5us) server_negotiation.cc:407] Connection header received
0811 05:13:20.518311 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:20.518312 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:20.518324 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:20.518370 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:13:20.518545 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:20.519051 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:20.519808 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:20.520067 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:20.520153 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:20.520276 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:13:20.520313 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:13:20.520364 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:20.520422 (+ 58us) server_negotiation.cc:299] Negotiation successful
0811 05:13:20.520447 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260811 05:13:20.520553 11785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:20.518113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:41671 (local address 127.28.143.131:33179)
0811 05:13:20.518156 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:20.518170 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:20.518215 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:20.518386 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:20.518388 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:20.518389 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 05:13:20.518442 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:20.518443 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:20.519120 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:20.519121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:20.519701 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:20.519703 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:20.520152 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:20.520153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:20.520198 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:20.520205 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 05:13:20.520365 (+ 160us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 05:13:20.520368 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 05:13:20.520418 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 05:13:20.520474 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260811 05:13:20.531450 10695 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:20.531566 10695 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:20.532248 10695 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4b9d89233cf04d7697a4dd645a3893ae: Sending DeleteTablet for 3 replicas of tablet 21c4bdf7394e405aabc2a436c49ef73f
I20260811 05:13:20.532542 11518 tablet_service.cc:1514] Processing DeleteTablet for tablet 21c4bdf7394e405aabc2a436c49ef73f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:20 UTC) from {username='oryx', principal='oryx/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:36154
I20260811 05:13:20.532541 11246 tablet_service.cc:1514] Processing DeleteTablet for tablet 21c4bdf7394e405aabc2a436c49ef73f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:20 UTC) from {username='oryx', principal='oryx/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:42342
I20260811 05:13:20.532536 10967 tablet_service.cc:1514] Processing DeleteTablet for tablet 21c4bdf7394e405aabc2a436c49ef73f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:20 UTC) from {username='oryx', principal='oryx/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:48010
I20260811 05:13:20.532666 11740 tablet_replica.cc:330] T 21c4bdf7394e405aabc2a436c49ef73f P a0c7e62d8f73465585819096c63c4a53: stopping tablet replica
I20260811 05:13:20.532668 11741 tablet_replica.cc:330] T 21c4bdf7394e405aabc2a436c49ef73f P 9dbce36fab4e4dd695c4d4d879e301c5: stopping tablet replica
I20260811 05:13:20.532673 11742 tablet_replica.cc:330] T 21c4bdf7394e405aabc2a436c49ef73f P 8f7748967c7344fca44f6bfe49a720de: stopping tablet replica
I20260811 05:13:20.532716 11742 raft_consensus.cc:2229] T 21c4bdf7394e405aabc2a436c49ef73f P 8f7748967c7344fca44f6bfe49a720de [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:13:20.532723 11741 raft_consensus.cc:2229] T 21c4bdf7394e405aabc2a436c49ef73f P 9dbce36fab4e4dd695c4d4d879e301c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:20.532727 11740 raft_consensus.cc:2229] T 21c4bdf7394e405aabc2a436c49ef73f P a0c7e62d8f73465585819096c63c4a53 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:20.532761 11741 raft_consensus.cc:2258] T 21c4bdf7394e405aabc2a436c49ef73f P 9dbce36fab4e4dd695c4d4d879e301c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:20.532765 11740 raft_consensus.cc:2258] T 21c4bdf7394e405aabc2a436c49ef73f P a0c7e62d8f73465585819096c63c4a53 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:20.532799 11742 raft_consensus.cc:2258] T 21c4bdf7394e405aabc2a436c49ef73f P 8f7748967c7344fca44f6bfe49a720de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:20.533120 11741 ts_tablet_manager.cc:1905] T 21c4bdf7394e405aabc2a436c49ef73f P 9dbce36fab4e4dd695c4d4d879e301c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:20.533164 11740 ts_tablet_manager.cc:1905] T 21c4bdf7394e405aabc2a436c49ef73f P a0c7e62d8f73465585819096c63c4a53: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:20.534801 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 10836
I20260811 05:13:20.534814 11742 ts_tablet_manager.cc:1905] T 21c4bdf7394e405aabc2a436c49ef73f P 8f7748967c7344fca44f6bfe49a720de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:20.536301 11742 ts_tablet_manager.cc:1918] T 21c4bdf7394e405aabc2a436c49ef73f P 8f7748967c7344fca44f6bfe49a720de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 05:13:20.536366 11742 log.cc:1196] T 21c4bdf7394e405aabc2a436c49ef73f P 8f7748967c7344fca44f6bfe49a720de: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-1/wal/wals/21c4bdf7394e405aabc2a436c49ef73f
I20260811 05:13:20.537010 11742 ts_tablet_manager.cc:1939] T 21c4bdf7394e405aabc2a436c49ef73f P 8f7748967c7344fca44f6bfe49a720de: Deleting consensus metadata
I20260811 05:13:20.553146 11740 ts_tablet_manager.cc:1918] T 21c4bdf7394e405aabc2a436c49ef73f P a0c7e62d8f73465585819096c63c4a53: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 05:13:20.553234 11740 log.cc:1196] T 21c4bdf7394e405aabc2a436c49ef73f P a0c7e62d8f73465585819096c63c4a53: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425159060450-29246-0/minicluster-data/ts-2/wal/wals/21c4bdf7394e405aabc2a436c49ef73f
I20260811 05:13:20.553720 11740 ts_tablet_manager.cc:1939] T 21c4bdf7394e405aabc2a436c49ef73f P a0c7e62d8f73465585819096c63c4a53: Deleting consensus metadata
I20260811 05:13:20.554188 10674 catalog_manager.cc:4841] TS a0c7e62d8f73465585819096c63c4a53 (127.28.143.131:40121): tablet 21c4bdf7394e405aabc2a436c49ef73f (table test-table [id=50cad0fe362149a7aa2dd9ccfe47188c]) successfully deleted
I20260811 05:13:20.556697 10674 catalog_manager.cc:4841] TS 8f7748967c7344fca44f6bfe49a720de (127.28.143.130:41671): tablet 21c4bdf7394e405aabc2a436c49ef73f (table test-table [id=50cad0fe362149a7aa2dd9ccfe47188c]) successfully deleted
W20260811 05:13:20.565311 10674 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.143.129:37049 (error 108)
I20260811 05:13:20.565659 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 11111
I20260811 05:13:20.571478 11869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:20.565774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:37049 (local address 127.0.0.1:48020)
0811 05:13:20.571312 (+ 5538us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:20.571421 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.129:37049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5506,"thread_start_us":51,"threads_started":1}
W20260811 05:13:20.571619 10674 catalog_manager.cc:4584] TS 9dbce36fab4e4dd695c4d4d879e301c5 (127.28.143.129:37049): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 21c4bdf7394e405aabc2a436c49ef73f: Network error: Client connection negotiation failed: client connection to 127.28.143.129:37049: connect: Connection refused (error 111)
I20260811 05:13:20.584038 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 11380
I20260811 05:13:20.607106 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 10635
2026-08-11T05:13:20Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3247 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786425159060450-29246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:13:20 dist-test-slave-f7th krb5kdc[11878](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:13:20 dist-test-slave-f7th krb5kdc[11878](info): set up 2 sockets
Aug 11 05:13:20 dist-test-slave-f7th krb5kdc[11878](info): commencing operation
krb5kdc: starting...
W20260811 05:13:21.846648 29246 mini_kdc.cc:121] Time spent starting KDC: real 1.201s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Aug 11 05:13:21 dist-test-slave-f7th krb5kdc[11878](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:13:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:13:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
I20260811 05:13:21.985585 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:39039
--webserver_interface=127.28.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:41661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:39039
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:22.074564 11906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:22.074820 11906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:22.074853 11906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:22.076139 11906 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:22.076196 11906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:22.076215 11906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:22.076231 11906 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:22.076246 11906 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:13:22.077925 11906 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:41661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:39039
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:39039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11906
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:22.078253 11906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:22.078492 11906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:22.083846 11918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:22.083846 11915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:22.083851 11916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:22.084203 11906 server_base.cc:1034] running on GCE node
I20260811 05:13:22.084452 11906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:22.084781 11906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:22.085955 11906 hybrid_clock.cc:648] HybridClock initialized: now 1786425202085934 us; error 37 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T05:13:22Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1470 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:13:22 dist-test-slave-f7th krb5kdc[11928](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:13:22 dist-test-slave-f7th krb5kdc[11928](info): set up 2 sockets
Aug 11 05:13:22 dist-test-slave-f7th krb5kdc[11928](info): commencing operation
krb5kdc: starting...
W20260811 05:13:23.340349 29246 mini_kdc.cc:121] Time spent starting KDC: real 1.224s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Aug 11 05:13:23 dist-test-slave-f7th krb5kdc[11928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:13:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:13:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
I20260811 05:13:23.482319 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:39667
--webserver_interface=127.28.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:40471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:39667
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:23.574141 11944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:23.574421 11944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:23.574448 11944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:23.575723 11944 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:23.575776 11944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:23.575789 11944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:23.575801 11944 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:23.575814 11944 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:13:23.577616 11944 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:40471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:39667
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:39667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:23.577908 11944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:23.578128 11944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:23.584152 11953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:23.584179 11956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:23.584167 11958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:23.584578 11944 server_base.cc:1034] running on GCE node
I20260811 05:13:23.584904 11944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:23.585238 11944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:23.586449 11944 hybrid_clock.cc:648] HybridClock initialized: now 1786425203586426 us; error 41 us; skew 500 ppm
Aug 11 05:13:23 dist-test-slave-f7th krb5kdc[11928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425203, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:23.589354 11944 init.cc:377] Logged in from keytab as kudu/127.28.143.190@KRBTEST.COM (short username kudu)
I20260811 05:13:23.589850 11944 webserver.cc:460] Webserver started at http://127.28.143.190:34475/ using document root <none> and password file <none>
I20260811 05:13:23.590109 11944 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:23.590162 11944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:23.590289 11944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:23.591463 11944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "4fccf098c38f4139b2eef1e243abbb15"
format_stamp: "Formatted at 2026-08-11 05:13:23 on dist-test-slave-f7th"
server_key: "5e232160ee3bc1694c09be2c2f7e462c"
server_key_iv: "dc19ec415a6c9dd79cc25bae782ebe36"
server_key_version: "encryptionkey@0"
I20260811 05:13:23.591826 11944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "4fccf098c38f4139b2eef1e243abbb15"
format_stamp: "Formatted at 2026-08-11 05:13:23 on dist-test-slave-f7th"
server_key: "5e232160ee3bc1694c09be2c2f7e462c"
server_key_iv: "dc19ec415a6c9dd79cc25bae782ebe36"
server_key_version: "encryptionkey@0"
I20260811 05:13:23.593369 11944 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 05:13:23.594235 11975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:23.594496 11944 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 05:13:23.594600 11944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "4fccf098c38f4139b2eef1e243abbb15"
format_stamp: "Formatted at 2026-08-11 05:13:23 on dist-test-slave-f7th"
server_key: "5e232160ee3bc1694c09be2c2f7e462c"
server_key_iv: "dc19ec415a6c9dd79cc25bae782ebe36"
server_key_version: "encryptionkey@0"
I20260811 05:13:23.594671 11944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:23.628811 11944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:23.631335 11944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:23.631503 11944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:23.635924 11944 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:39667
I20260811 05:13:23.636023 12069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:39667 every 8 connection(s)
I20260811 05:13:23.636394 11944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:13:23.637404 12070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:23.639937 12070 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fccf098c38f4139b2eef1e243abbb15: Bootstrap starting.
I20260811 05:13:23.640491 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 11944
I20260811 05:13:23.640605 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:13:23.640764 12070 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4fccf098c38f4139b2eef1e243abbb15: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:23.640820 29246 external_mini_cluster.cc:1428] Setting key 74090b4ac411eb436623940605546c06
I20260811 05:13:23.641099 12070 log.cc:824] T 00000000000000000000000000000000 P 4fccf098c38f4139b2eef1e243abbb15: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:23.642040 12070 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fccf098c38f4139b2eef1e243abbb15: No bootstrap required, opened a new log
I20260811 05:13:23.644488 12070 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4fccf098c38f4139b2eef1e243abbb15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fccf098c38f4139b2eef1e243abbb15" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39667 } }
I20260811 05:13:23.644732 12070 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4fccf098c38f4139b2eef1e243abbb15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:23.644776 12070 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4fccf098c38f4139b2eef1e243abbb15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fccf098c38f4139b2eef1e243abbb15, State: Initialized, Role: FOLLOWER
I20260811 05:13:23.644966 12070 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4fccf098c38f4139b2eef1e243abbb15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fccf098c38f4139b2eef1e243abbb15" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39667 } }
I20260811 05:13:23.645057 12070 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4fccf098c38f4139b2eef1e243abbb15 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:13:23.645081 12070 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4fccf098c38f4139b2eef1e243abbb15 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:13:23.645134 12070 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4fccf098c38f4139b2eef1e243abbb15 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:23.645876 12070 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4fccf098c38f4139b2eef1e243abbb15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fccf098c38f4139b2eef1e243abbb15" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39667 } }
I20260811 05:13:23.645998 12070 leader_election.cc:302] T 00000000000000000000000000000000 P 4fccf098c38f4139b2eef1e243abbb15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4fccf098c38f4139b2eef1e243abbb15; no voters:
I20260811 05:13:23.646214 12070 leader_election.cc:288] T 00000000000000000000000000000000 P 4fccf098c38f4139b2eef1e243abbb15 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:13:23.646286 12080 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4fccf098c38f4139b2eef1e243abbb15 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:23.646514 12070 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4fccf098c38f4139b2eef1e243abbb15 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:13:23.646543 12080 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4fccf098c38f4139b2eef1e243abbb15 [term 1 LEADER]: Becoming Leader. State: Replica: 4fccf098c38f4139b2eef1e243abbb15, State: Running, Role: LEADER
I20260811 05:13:23.646669 12080 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4fccf098c38f4139b2eef1e243abbb15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fccf098c38f4139b2eef1e243abbb15" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39667 } }
I20260811 05:13:23.647064 12082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fccf098c38f4139b2eef1e243abbb15 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4fccf098c38f4139b2eef1e243abbb15. Latest consensus state: current_term: 1 leader_uuid: "4fccf098c38f4139b2eef1e243abbb15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fccf098c38f4139b2eef1e243abbb15" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39667 } } }
I20260811 05:13:23.647150 12082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fccf098c38f4139b2eef1e243abbb15 [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:23.647424 12084 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:13:23.647511 12081 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fccf098c38f4139b2eef1e243abbb15 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4fccf098c38f4139b2eef1e243abbb15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fccf098c38f4139b2eef1e243abbb15" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39667 } } }
I20260811 05:13:23.647670 12081 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fccf098c38f4139b2eef1e243abbb15 [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:23.648061 12084 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 11 05:13:23 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:13:23.650187 12084 catalog_manager.cc:1347] Generated new cluster ID: ce305e516138424f86e52c29b26147b4
I20260811 05:13:23.650247 12084 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:13:23.652844 12078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:23.641446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47956 (local address 127.28.143.190:39667)
0811 05:13:23.641643 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:23.641654 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:23.641691 (+ 37us) server_negotiation.cc:407] Connection header received
0811 05:13:23.642326 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:23.642336 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:23.642421 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:23.642589 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:23.643276 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:23.644711 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:23.645666 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:23.646293 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:23.646421 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:23.649563 (+ 3142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:23.649568 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:23.649568 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:23.649582 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:23.651244 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:23.651612 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:23.651616 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:23.651616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:23.651708 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:23.651896 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:23.651897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:23.651897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:23.652154 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:23.652307 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:23.652439 (+ 132us) server_negotiation.cc:299] Negotiation successful
0811 05:13:23.652587 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":130,"thread_start_us":76,"threads_started":1}
I20260811 05:13:23.678946 12084 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:13:23.679520 12084 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:13:23.683032 12084 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4fccf098c38f4139b2eef1e243abbb15: Generated new TSK 0
I20260811 05:13:23.683254 12084 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.143.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab.
Entry for principal kudu/127.28.143.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab.
I20260811 05:13:23.723984 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.129:0
--local_ip_for_outbound_sockets=127.28.143.129
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:39667
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:40471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:23.828241 12117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:23.828509 12117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:23.828537 12117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:23.829974 12117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:23.830047 12117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:23.830077 12117 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.129
I20260811 05:13:23.831909 12117 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:40471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:39667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.129
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:23.832172 12117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:23.832432 12117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:23.838433 12129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:23.838606 12127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:23.838613 12131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:23.839255 12117 server_base.cc:1034] running on GCE node
I20260811 05:13:23.839534 12117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:23.839833 12117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:23.841007 12117 hybrid_clock.cc:648] HybridClock initialized: now 1786425203840973 us; error 97 us; skew 500 ppm
Aug 11 05:13:23 dist-test-slave-f7th krb5kdc[11928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425203, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:23.844475 12117 init.cc:377] Logged in from keytab as kudu/127.28.143.129@KRBTEST.COM (short username kudu)
I20260811 05:13:23.845101 12117 webserver.cc:460] Webserver started at http://127.28.143.129:33647/ using document root <none> and password file <none>
I20260811 05:13:23.845363 12117 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:23.845425 12117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:23.845592 12117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:23.847083 12117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-0/data/instance:
uuid: "bec02d19da574594ac4965cfd011a618"
format_stamp: "Formatted at 2026-08-11 05:13:23 on dist-test-slave-f7th"
server_key: "437a81479eca191b3c5c8d9437b08a6e"
server_key_iv: "f69ea56c32e9d7524c9759601be0d923"
server_key_version: "encryptionkey@0"
I20260811 05:13:23.848031 12117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance:
uuid: "bec02d19da574594ac4965cfd011a618"
format_stamp: "Formatted at 2026-08-11 05:13:23 on dist-test-slave-f7th"
server_key: "437a81479eca191b3c5c8d9437b08a6e"
server_key_iv: "f69ea56c32e9d7524c9759601be0d923"
server_key_version: "encryptionkey@0"
I20260811 05:13:23.849879 12117 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260811 05:13:23.851143 12139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:23.851902 12117 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 05:13:23.852190 12117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-0/wal
uuid: "bec02d19da574594ac4965cfd011a618"
format_stamp: "Formatted at 2026-08-11 05:13:23 on dist-test-slave-f7th"
server_key: "437a81479eca191b3c5c8d9437b08a6e"
server_key_iv: "f69ea56c32e9d7524c9759601be0d923"
server_key_version: "encryptionkey@0"
I20260811 05:13:23.852409 12117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:23.871721 12117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:23.874755 12117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:23.875120 12117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:23.875408 12117 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:23.875775 12117 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:23.875813 12117 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:23.875849 12117 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:23.875867 12117 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:23.882699 12117 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.129:40021
I20260811 05:13:23.882772 12342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.129:40021 every 8 connection(s)
I20260811 05:13:23.883149 12117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
I20260811 05:13:23.892531 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 12117
I20260811 05:13:23.892665 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance
I20260811 05:13:23.892853 29246 external_mini_cluster.cc:1428] Setting key 6950ab6db4e033311676a7be1d9aa044
Aug 11 05:13:23 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425203, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:13:23.897969 12078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:23.884024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:33059 (local address 127.28.143.190:39667)
0811 05:13:23.884095 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:23.884098 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:23.884287 (+ 189us) server_negotiation.cc:407] Connection header received
0811 05:13:23.884884 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:23.884885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:23.884897 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:23.884975 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:23.886194 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:23.886748 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:23.887971 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:23.888202 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:23.888272 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:23.894411 (+ 6139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:23.894413 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:23.894413 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:23.894423 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:23.896165 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:23.896747 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:23.896749 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:23.896750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:23.896832 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:23.897223 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:23.897225 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:23.897225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:23.897383 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:23.897454 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:23.897764 (+ 310us) server_negotiation.cc:299] Negotiation successful
0811 05:13:23.897871 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 05:13:23.898114 12347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:23.883956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.129:33059)
0811 05:13:23.884156 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:23.884204 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:23.884775 (+ 571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:23.885138 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:23.885143 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:23.885782 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:23.886002 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:23.886008 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:23.886943 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:23.886944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:23.887808 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:23.887811 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:23.888353 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:23.888354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:23.888568 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:23.892293 (+ 3725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:23.892308 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:23.894259 (+ 1951us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:23.896302 (+ 2043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:23.896313 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:23.896316 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:23.896605 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:23.896942 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:23.896944 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:23.896944 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:23.897144 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:23.897445 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:23.897449 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:23.897609 (+ 160us) client_negotiation.cc:769] Sending connection context
0811 05:13:23.897710 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 05:13:23.897870 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":97,"threads_started":1}
I20260811 05:13:23.898838 12344 heartbeater.cc:347] Connected to a master server at 127.28.143.190:39667
I20260811 05:13:23.898983 12344 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:23.899191 12344 heartbeater.cc:511] Master 127.28.143.190:39667 requested a full tablet report, sending...
I20260811 05:13:23.899840 12005 ts_manager.cc:194] Registered new tserver with Master: bec02d19da574594ac4965cfd011a618 (127.28.143.129:40021)
I20260811 05:13:23.900523 12005 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.129@KRBTEST.COM'} at 127.28.143.129:33059
WARNING: no policy specified for kudu/127.28.143.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab.
Entry for principal kudu/127.28.143.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab.
I20260811 05:13:23.928664 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.130:0
--local_ip_for_outbound_sockets=127.28.143.130
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:39667
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:40471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:24.018802 12353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:24.019039 12353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:24.019064 12353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:24.020291 12353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:24.020339 12353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:24.020366 12353 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.130
I20260811 05:13:24.021994 12353 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:40471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:39667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.130
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:24.022250 12353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:24.022493 12353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:24.027905 12368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:24.027915 12360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:24.027913 12363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:24.028185 12353 server_base.cc:1034] running on GCE node
I20260811 05:13:24.028399 12353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:24.028683 12353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:24.029863 12353 hybrid_clock.cc:648] HybridClock initialized: now 1786425204029847 us; error 27 us; skew 500 ppm
Aug 11 05:13:24 dist-test-slave-f7th krb5kdc[11928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425204, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:24.032697 12353 init.cc:377] Logged in from keytab as kudu/127.28.143.130@KRBTEST.COM (short username kudu)
I20260811 05:13:24.033278 12353 webserver.cc:460] Webserver started at http://127.28.143.130:34403/ using document root <none> and password file <none>
I20260811 05:13:24.033535 12353 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:24.033588 12353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:24.033717 12353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:24.034864 12353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-1/data/instance:
uuid: "f066ac1d452d4c1f81a83d34dcc99346"
format_stamp: "Formatted at 2026-08-11 05:13:24 on dist-test-slave-f7th"
server_key: "ddd82113f4bd2917f2b6abc1d8e99205"
server_key_iv: "22c9e8200ff3c5c6a732bcfdb60996af"
server_key_version: "encryptionkey@0"
I20260811 05:13:24.035341 12353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance:
uuid: "f066ac1d452d4c1f81a83d34dcc99346"
format_stamp: "Formatted at 2026-08-11 05:13:24 on dist-test-slave-f7th"
server_key: "ddd82113f4bd2917f2b6abc1d8e99205"
server_key_iv: "22c9e8200ff3c5c6a732bcfdb60996af"
server_key_version: "encryptionkey@0"
I20260811 05:13:24.036971 12353 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260811 05:13:24.037791 12380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:24.037997 12353 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:24.038097 12353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-1/wal
uuid: "f066ac1d452d4c1f81a83d34dcc99346"
format_stamp: "Formatted at 2026-08-11 05:13:24 on dist-test-slave-f7th"
server_key: "ddd82113f4bd2917f2b6abc1d8e99205"
server_key_iv: "22c9e8200ff3c5c6a732bcfdb60996af"
server_key_version: "encryptionkey@0"
I20260811 05:13:24.038166 12353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:24.046623 12353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:24.049151 12353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:24.049319 12353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:24.049569 12353 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:24.049896 12353 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:24.049932 12353 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:24.049962 12353 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:24.049980 12353 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:24.057502 12353 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.130:46303
I20260811 05:13:24.057559 12588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.130:46303 every 8 connection(s)
I20260811 05:13:24.057972 12353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
I20260811 05:13:24.065953 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 12353
I20260811 05:13:24.066063 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance
I20260811 05:13:24.066241 29246 external_mini_cluster.cc:1428] Setting key f7f20b39de97033dd89c81ebf2c3b82f
Aug 11 05:13:24 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425204, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:13:24.071201 12078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:24.059143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:60463 (local address 127.28.143.190:39667)
0811 05:13:24.059343 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:24.059347 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:24.059671 (+ 324us) server_negotiation.cc:407] Connection header received
0811 05:13:24.060196 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:24.060198 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:24.060213 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:24.060281 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:24.061658 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:24.062229 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:24.063598 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:24.063868 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:24.063989 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:24.067085 (+ 3096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:24.067086 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:24.067087 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:24.067096 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:24.068602 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:24.069458 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:24.069459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:24.069460 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:24.069694 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:24.070048 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:24.070050 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:24.070050 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:24.070343 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:24.070423 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:24.070971 (+ 548us) server_negotiation.cc:299] Negotiation successful
0811 05:13:24.071103 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260811 05:13:24.071440 12595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:24.058850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.130:60463)
0811 05:13:24.059443 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:24.059507 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:24.060075 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:24.060423 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:24.060428 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:24.061236 (+ 808us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:24.061486 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:24.061491 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:24.062363 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:24.062364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:24.063402 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:24.063406 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:24.064161 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:24.064167 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:24.064359 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:24.065083 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:24.065094 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:24.066942 (+ 1848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:24.068868 (+ 1926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:24.068880 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:24.068884 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:24.069344 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:24.069793 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:24.069794 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:24.069794 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:24.069942 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:24.070426 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:24.070436 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:24.070726 (+ 290us) client_negotiation.cc:769] Sending connection context
0811 05:13:24.070934 (+ 208us) client_negotiation.cc:240] Negotiation successful
0811 05:13:24.071144 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":451,"threads_started":1}
I20260811 05:13:24.071978 12593 heartbeater.cc:347] Connected to a master server at 127.28.143.190:39667
I20260811 05:13:24.072117 12593 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:24.072376 12593 heartbeater.cc:511] Master 127.28.143.190:39667 requested a full tablet report, sending...
I20260811 05:13:24.072896 12005 ts_manager.cc:194] Registered new tserver with Master: f066ac1d452d4c1f81a83d34dcc99346 (127.28.143.130:46303)
I20260811 05:13:24.073338 12005 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.130@KRBTEST.COM'} at 127.28.143.130:60463
WARNING: no policy specified for kudu/127.28.143.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab.
Entry for principal kudu/127.28.143.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab.
I20260811 05:13:24.097069 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.131:0
--local_ip_for_outbound_sockets=127.28.143.131
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:39667
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:40471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:24.193143 12600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:24.193415 12600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:24.193461 12600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:24.195003 12600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:24.195082 12600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:24.195204 12600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.131
I20260811 05:13:24.197197 12600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:40471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:39667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.131
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:24.197481 12600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:24.197722 12600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:24.203207 12611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:24.203182 12609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:24.203338 12600 server_base.cc:1034] running on GCE node
W20260811 05:13:24.203176 12613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:24.203696 12600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:24.203959 12600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:24.205147 12600 hybrid_clock.cc:648] HybridClock initialized: now 1786425204205131 us; error 32 us; skew 500 ppm
Aug 11 05:13:24 dist-test-slave-f7th krb5kdc[11928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425204, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:24.208078 12600 init.cc:377] Logged in from keytab as kudu/127.28.143.131@KRBTEST.COM (short username kudu)
I20260811 05:13:24.208591 12600 webserver.cc:460] Webserver started at http://127.28.143.131:35547/ using document root <none> and password file <none>
I20260811 05:13:24.208865 12600 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:24.208920 12600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:24.209051 12600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:24.210143 12600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-2/data/instance:
uuid: "805c4884c1cb41ba8d925e66b9299519"
format_stamp: "Formatted at 2026-08-11 05:13:24 on dist-test-slave-f7th"
server_key: "6b0460b8ee13d5233c2c514356cc5235"
server_key_iv: "34948fc5a6d97498a1e4c4b6f61759b4"
server_key_version: "encryptionkey@0"
I20260811 05:13:24.210491 12600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance:
uuid: "805c4884c1cb41ba8d925e66b9299519"
format_stamp: "Formatted at 2026-08-11 05:13:24 on dist-test-slave-f7th"
server_key: "6b0460b8ee13d5233c2c514356cc5235"
server_key_iv: "34948fc5a6d97498a1e4c4b6f61759b4"
server_key_version: "encryptionkey@0"
I20260811 05:13:24.211984 12600 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 05:13:24.213030 12623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:24.213264 12600 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 05:13:24.213370 12600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-2/wal
uuid: "805c4884c1cb41ba8d925e66b9299519"
format_stamp: "Formatted at 2026-08-11 05:13:24 on dist-test-slave-f7th"
server_key: "6b0460b8ee13d5233c2c514356cc5235"
server_key_iv: "34948fc5a6d97498a1e4c4b6f61759b4"
server_key_version: "encryptionkey@0"
I20260811 05:13:24.213472 12600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:24.240326 12600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:24.243340 12600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:24.243497 12600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:24.243750 12600 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:24.244091 12600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:24.244127 12600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:24.244149 12600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:24.244168 12600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:24.250401 12600 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.131:35903
I20260811 05:13:24.250577 12813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.131:35903 every 8 connection(s)
I20260811 05:13:24.250828 12600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
I20260811 05:13:24.254211 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 12600
I20260811 05:13:24.254323 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance
I20260811 05:13:24.254513 29246 external_mini_cluster.cc:1428] Setting key 412e4a92c439ff0916067b697ce6781f
Aug 11 05:13:24 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425204, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:13:24.265590 12821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:24.251614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.131:55587)
0811 05:13:24.252051 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:24.252098 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:24.252607 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:24.253284 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:24.253288 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:24.253973 (+ 685us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:24.254289 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:24.254297 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:24.255443 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:24.255444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:24.256740 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:24.256746 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:24.257231 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:24.257233 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:24.257394 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:24.258197 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:24.258215 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:24.260276 (+ 2061us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:24.263061 (+ 2785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:24.263077 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:24.263081 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:24.263426 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:24.263839 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:24.263841 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:24.263842 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:24.264127 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:24.264681 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:24.264699 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:24.264936 (+ 237us) client_negotiation.cc:769] Sending connection context
0811 05:13:24.265091 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 05:13:24.265341 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":249,"threads_started":1}
I20260811 05:13:24.265646 12078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:24.251943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:55587 (local address 127.28.143.190:39667)
0811 05:13:24.252058 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:24.252060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:24.252360 (+ 300us) server_negotiation.cc:407] Connection header received
0811 05:13:24.253024 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:24.253025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:24.253042 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:24.253190 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:24.254470 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:24.255236 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:24.256868 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:24.257144 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:24.257216 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:24.260460 (+ 3244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:24.260462 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:24.260463 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:24.260473 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:24.262782 (+ 2309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:24.263572 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:24.263575 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:24.263575 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:24.263664 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:24.264272 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:24.264274 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:24.264274 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:24.264464 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:24.264561 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:24.265234 (+ 673us) server_negotiation.cc:299] Negotiation successful
0811 05:13:24.265559 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 05:13:24.266564 12817 heartbeater.cc:347] Connected to a master server at 127.28.143.190:39667
I20260811 05:13:24.266813 12817 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:24.267161 12817 heartbeater.cc:511] Master 127.28.143.190:39667 requested a full tablet report, sending...
I20260811 05:13:24.267768 12003 ts_manager.cc:194] Registered new tserver with Master: 805c4884c1cb41ba8d925e66b9299519 (127.28.143.131:35903)
I20260811 05:13:24.268206 12003 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.131@KRBTEST.COM'} at 127.28.143.131:55587
I20260811 05:13:24.278541 29246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.28.143.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.143.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.143.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.129.keytab.
Entry for principal oryx/127.28.143.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.129.keytab.
WARNING: no policy specified for oryx/127.28.143.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.143.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.143.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.130.keytab.
Entry for principal oryx/127.28.143.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.130.keytab.
WARNING: no policy specified for oryx/127.28.143.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.143.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.143.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.131.keytab.
Entry for principal oryx/127.28.143.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.131.keytab.
I20260811 05:13:24.380753 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 12117
I20260811 05:13:24.386438 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 12353
I20260811 05:13:24.456099 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 12600
I20260811 05:13:24.469081 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 11944
I20260811 05:13:24.486219 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:39667
--webserver_interface=127.28.143.190
--webserver_port=34475
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:40471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:39667
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:24.575673 12831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:24.575914 12831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:24.575939 12831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:24.577267 12831 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:24.577329 12831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:24.577343 12831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:24.577355 12831 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:24.577366 12831 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:13:24.579092 12831 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:40471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:39667
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:39667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=34475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:24.579384 12831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:24.579633 12831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:24.585361 12841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:24.585361 12855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:24.585393 12831 server_base.cc:1034] running on GCE node
W20260811 05:13:24.585366 12844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:24.586056 12831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:24.586398 12831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:24.587594 12831 hybrid_clock.cc:648] HybridClock initialized: now 1786425204587574 us; error 40 us; skew 500 ppm
Aug 11 05:13:24 dist-test-slave-f7th krb5kdc[11928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425204, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:24.590792 12831 init.cc:377] Logged in from keytab as kudu/127.28.143.190@KRBTEST.COM (short username kudu)
I20260811 05:13:24.591377 12831 webserver.cc:460] Webserver started at http://127.28.143.190:34475/ using document root <none> and password file <none>
I20260811 05:13:24.591701 12831 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:24.591786 12831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:24.593701 12831 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:24.594919 12864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:24.595230 12831 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:24.595369 12831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "4fccf098c38f4139b2eef1e243abbb15"
format_stamp: "Formatted at 2026-08-11 05:13:23 on dist-test-slave-f7th"
server_key: "5e232160ee3bc1694c09be2c2f7e462c"
server_key_iv: "dc19ec415a6c9dd79cc25bae782ebe36"
server_key_version: "encryptionkey@0"
I20260811 05:13:24.595773 12831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:24.609946 12831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:24.612502 12831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:24.612715 12831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:24.617758 12831 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:39667
I20260811 05:13:24.617823 12958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:39667 every 8 connection(s)
I20260811 05:13:24.618269 12831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:13:24.620090 12959 sys_catalog.cc:263] Verifying existing consensus state
I20260811 05:13:24.620883 12959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fccf098c38f4139b2eef1e243abbb15: Bootstrap starting.
I20260811 05:13:24.622841 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 12831
I20260811 05:13:24.623201 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.129:40021
--local_ip_for_outbound_sockets=127.28.143.129
--tserver_master_addrs=127.28.143.190:39667
--webserver_port=33647
--webserver_interface=127.28.143.129
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:40471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.28.143.129
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:13:24.630931 12959 log.cc:824] T 00000000000000000000000000000000 P 4fccf098c38f4139b2eef1e243abbb15: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:24.633152 12959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fccf098c38f4139b2eef1e243abbb15: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 05:13:24.633579 12959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fccf098c38f4139b2eef1e243abbb15: Bootstrap complete.
I20260811 05:13:24.636039 12959 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4fccf098c38f4139b2eef1e243abbb15 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fccf098c38f4139b2eef1e243abbb15" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39667 } }
I20260811 05:13:24.636314 12959 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4fccf098c38f4139b2eef1e243abbb15 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fccf098c38f4139b2eef1e243abbb15, State: Initialized, Role: FOLLOWER
I20260811 05:13:24.636483 12959 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4fccf098c38f4139b2eef1e243abbb15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fccf098c38f4139b2eef1e243abbb15" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39667 } }
I20260811 05:13:24.636559 12959 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4fccf098c38f4139b2eef1e243abbb15 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:13:24.636603 12959 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4fccf098c38f4139b2eef1e243abbb15 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:13:24.636670 12959 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4fccf098c38f4139b2eef1e243abbb15 [term 1 FOLLOWER]: Advancing to term 2
I20260811 05:13:24.638120 12959 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4fccf098c38f4139b2eef1e243abbb15 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fccf098c38f4139b2eef1e243abbb15" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39667 } }
I20260811 05:13:24.638269 12959 leader_election.cc:302] T 00000000000000000000000000000000 P 4fccf098c38f4139b2eef1e243abbb15 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4fccf098c38f4139b2eef1e243abbb15; no voters:
I20260811 05:13:24.638667 12959 leader_election.cc:288] T 00000000000000000000000000000000 P 4fccf098c38f4139b2eef1e243abbb15 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 05:13:24.638728 12964 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4fccf098c38f4139b2eef1e243abbb15 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 05:13:24.638929 12964 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4fccf098c38f4139b2eef1e243abbb15 [term 2 LEADER]: Becoming Leader. State: Replica: 4fccf098c38f4139b2eef1e243abbb15, State: Running, Role: LEADER
I20260811 05:13:24.639043 12959 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4fccf098c38f4139b2eef1e243abbb15 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:13:24.639052 12964 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4fccf098c38f4139b2eef1e243abbb15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fccf098c38f4139b2eef1e243abbb15" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39667 } }
I20260811 05:13:24.639634 12968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fccf098c38f4139b2eef1e243abbb15 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4fccf098c38f4139b2eef1e243abbb15. Latest consensus state: current_term: 2 leader_uuid: "4fccf098c38f4139b2eef1e243abbb15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fccf098c38f4139b2eef1e243abbb15" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39667 } } }
I20260811 05:13:24.639669 12966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fccf098c38f4139b2eef1e243abbb15 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4fccf098c38f4139b2eef1e243abbb15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fccf098c38f4139b2eef1e243abbb15" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39667 } } }
I20260811 05:13:24.639750 12968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fccf098c38f4139b2eef1e243abbb15 [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:24.639806 12966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fccf098c38f4139b2eef1e243abbb15 [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:24.640275 12979 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:13:24.641186 12979 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:13:24.641340 12979 catalog_manager.cc:1259] Loaded cluster ID: ce305e516138424f86e52c29b26147b4
I20260811 05:13:24.641379 12979 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:13:24.642647 12979 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:13:24.642922 12979 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4fccf098c38f4139b2eef1e243abbb15: Loaded TSK: 0
I20260811 05:13:24.643173 12979 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 05:13:24.721302 12961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:24.721544 12961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:24.721581 12961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:24.722890 12961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:24.722944 12961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:24.722982 12961 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.129
I20260811 05:13:24.724582 12961 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:40471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.129.keytab
--principal=oryx/127.28.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.129:40021
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.143.129
--webserver_port=33647
--tserver_master_addrs=127.28.143.190:39667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12961
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.129
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:24.724882 12961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:24.725123 12961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:24.730499 12999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:24.730499 13008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:24.730499 13005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:24.731276 12961 server_base.cc:1034] running on GCE node
I20260811 05:13:24.731519 12961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:24.731772 12961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:24.732959 12961 hybrid_clock.cc:648] HybridClock initialized: now 1786425204732934 us; error 39 us; skew 500 ppm
Aug 11 05:13:24 dist-test-slave-f7th krb5kdc[11928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425204, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:24.735713 12961 init.cc:377] Logged in from keytab as oryx/127.28.143.129@KRBTEST.COM (short username oryx)
I20260811 05:13:24.736228 12961 webserver.cc:460] Webserver started at http://127.28.143.129:33647/ using document root <none> and password file <none>
I20260811 05:13:24.736488 12961 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:24.736562 12961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:24.737995 12961 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:24.738612 13019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:24.738797 12961 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:24.738895 12961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-0/wal
uuid: "bec02d19da574594ac4965cfd011a618"
format_stamp: "Formatted at 2026-08-11 05:13:23 on dist-test-slave-f7th"
server_key: "437a81479eca191b3c5c8d9437b08a6e"
server_key_iv: "f69ea56c32e9d7524c9759601be0d923"
server_key_version: "encryptionkey@0"
I20260811 05:13:24.739274 12961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:24.754004 12961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:24.756831 12961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:24.757062 12961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:24.757349 12961 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:24.757718 12961 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:24.757756 12961 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:24.757788 12961 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:24.757812 12961 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:24.765589 12961 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.129:40021
I20260811 05:13:24.765617 13211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.129:40021 every 8 connection(s)
I20260811 05:13:24.766036 12961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
I20260811 05:13:24.770928 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 12961
I20260811 05:13:24.771247 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.130:46303
--local_ip_for_outbound_sockets=127.28.143.130
--tserver_master_addrs=127.28.143.190:39667
--webserver_port=34403
--webserver_interface=127.28.143.130
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:40471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.28.143.130
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 05:13:24 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.129@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:24 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.129@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:24.778350 13215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:24.766825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.129:52041)
0811 05:13:24.767028 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:24.767082 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:24.767810 (+ 728us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:24.768519 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:24.768524 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:24.769377 (+ 853us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:24.769587 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:24.769594 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:24.771408 (+ 1814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:24.771409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:24.772803 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:24.772809 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:24.773935 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:24.773937 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:24.774104 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:24.774827 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:24.774842 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:24.778085 (+ 3243us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":92,"threads_started":1}
W20260811 05:13:24.778677 13218 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:24.766959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:52041 (local address 127.28.143.190:39667)
0811 05:13:24.768041 (+ 1082us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:24.768051 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:24.768091 (+ 40us) server_negotiation.cc:407] Connection header received
0811 05:13:24.768233 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:24.768240 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:24.768296 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:24.768437 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:24.769735 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:24.771249 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:24.772956 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:24.773647 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:24.773741 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:24.778304 (+ 4563us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.129:52041: BlockingRecv error: recv got EOF from 127.28.143.129:52041 (error 108)
Metrics: {"server-negotiator.queue_time_us":1022,"thread_start_us":91,"threads_started":1}
W20260811 05:13:24.778971 13212 heartbeater.cc:644] Failed to heartbeat to 127.28.143.190:39667 (0 consecutive failures): Not authorized: Failed to ping master at 127.28.143.190:39667: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Aug 11 05:13:24 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.129@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:24 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.129@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:24.785960 13215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:24.779277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.129:51539)
0811 05:13:24.779404 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:24.779416 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:24.779472 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:24.779668 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:24.779669 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:24.779860 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:24.779939 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:24.779942 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:24.781102 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:24.781104 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:24.781910 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:24.781914 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:24.782452 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:24.782453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:24.782538 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:24.783277 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:24.783283 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:24.785878 (+ 2595us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106}
W20260811 05:13:24.786154 13218 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:24.779354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:51539 (local address 127.28.143.190:39667)
0811 05:13:24.779423 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:24.779426 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:24.779457 (+ 31us) server_negotiation.cc:407] Connection header received
0811 05:13:24.779556 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:24.779557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:24.779575 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:24.779657 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:24.780025 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:24.780981 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:24.782026 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:24.782353 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:24.782446 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:24.786078 (+ 3632us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.129:51539: BlockingRecv error: recv got EOF from 127.28.143.129:51539 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 11 05:13:24 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.129@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:24 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.129@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:24.792510 13215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:24.786473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.129:56797)
0811 05:13:24.786524 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:24.786537 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:24.786602 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:24.786760 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:24.786761 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:24.786902 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:24.786947 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:24.786949 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:24.787985 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:24.787986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:24.788563 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:24.788566 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:24.789485 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:24.789487 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:24.789547 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:24.790135 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:24.790141 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:24.792440 (+ 2299us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260811 05:13:24.792716 13218 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:24.786558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:56797 (local address 127.28.143.190:39667)
0811 05:13:24.786628 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:24.786630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:24.786639 (+ 9us) server_negotiation.cc:407] Connection header received
0811 05:13:24.786669 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:24.786670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:24.786683 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:24.786755 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:24.787018 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:24.787834 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:24.788702 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:24.789180 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:24.789442 (+ 262us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:24.792607 (+ 3165us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.129:56797: BlockingRecv error: recv got EOF from 127.28.143.129:56797 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260811 05:13:24.792771 13212 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 05:13:24.865888 13222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:24.866171 13222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:24.866200 13222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:24.868677 13222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:24.868767 13222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:24.868801 13222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.130
I20260811 05:13:24.871059 13222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:40471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.130.keytab
--principal=oryx/127.28.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.130:46303
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.143.130
--webserver_port=34403
--tserver_master_addrs=127.28.143.190:39667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.130
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:24.871439 13222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:24.871716 13222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:24.878010 13230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:24.878010 13233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:24.878012 13235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:24.878062 13222 server_base.cc:1034] running on GCE node
I20260811 05:13:24.878614 13222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:24.878888 13222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:24.880096 13222 hybrid_clock.cc:648] HybridClock initialized: now 1786425204880077 us; error 37 us; skew 500 ppm
Aug 11 05:13:24 dist-test-slave-f7th krb5kdc[11928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425204, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:24.883459 13222 init.cc:377] Logged in from keytab as oryx/127.28.143.130@KRBTEST.COM (short username oryx)
I20260811 05:13:24.884064 13222 webserver.cc:460] Webserver started at http://127.28.143.130:34403/ using document root <none> and password file <none>
I20260811 05:13:24.884305 13222 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:24.884373 13222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:24.885994 13222 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:24.886722 13252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:24.886906 13222 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:24.887012 13222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-1/wal
uuid: "f066ac1d452d4c1f81a83d34dcc99346"
format_stamp: "Formatted at 2026-08-11 05:13:24 on dist-test-slave-f7th"
server_key: "ddd82113f4bd2917f2b6abc1d8e99205"
server_key_iv: "22c9e8200ff3c5c6a732bcfdb60996af"
server_key_version: "encryptionkey@0"
I20260811 05:13:24.887404 13222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:24.901440 13222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:24.903825 13222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:24.903980 13222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:24.904201 13222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:24.904556 13222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:24.904604 13222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:24.904686 13222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:24.904713 13222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:24.911628 13222 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.130:46303
I20260811 05:13:24.911686 13450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.130:46303 every 8 connection(s)
I20260811 05:13:24.912084 13222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
I20260811 05:13:24.918484 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 13222
I20260811 05:13:24.918784 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.131:35903
--local_ip_for_outbound_sockets=127.28.143.131
--tserver_master_addrs=127.28.143.190:39667
--webserver_port=35547
--webserver_interface=127.28.143.131
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:40471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.28.143.131
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 05:13:24 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.130@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:24 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.130@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:24.921262 13457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:24.913131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.130:40693)
0811 05:13:24.913322 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:24.913381 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:24.913885 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:24.914150 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:24.914153 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:24.914821 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:24.915040 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:24.915045 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:24.915880 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:24.915881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:24.916863 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:24.916867 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:24.917279 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:24.917280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:24.917438 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:24.918188 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:24.918199 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:24.921082 (+ 2883us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":79,"threads_started":1}
W20260811 05:13:24.921376 13218 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:24.913177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:40693 (local address 127.28.143.190:39667)
0811 05:13:24.913240 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:24.913242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:24.913493 (+ 251us) server_negotiation.cc:407] Connection header received
0811 05:13:24.914003 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:24.914005 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:24.914023 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:24.914091 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:24.915164 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:24.915755 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:24.916973 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:24.917171 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:24.917239 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:24.921310 (+ 4071us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.130:40693: BlockingRecv error: recv got EOF from 127.28.143.130:40693 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260811 05:13:24.921736 13455 heartbeater.cc:644] Failed to heartbeat to 127.28.143.190:39667 (0 consecutive failures): Not authorized: Failed to ping master at 127.28.143.190:39667: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Aug 11 05:13:24 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.130@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:24 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.130@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:24.926690 13457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:24.921990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.130:43749)
0811 05:13:24.922044 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:24.922056 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:24.922099 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:24.922239 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:24.922240 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:24.922383 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:24.922437 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:24.922439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:24.923247 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:24.923248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:24.923782 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:24.923783 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:24.924267 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:24.924268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:24.924341 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:24.924847 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:24.924852 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:24.926612 (+ 1760us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260811 05:13:24.926831 13218 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:24.922058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:43749 (local address 127.28.143.190:39667)
0811 05:13:24.922128 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:24.922130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:24.922139 (+ 9us) server_negotiation.cc:407] Connection header received
0811 05:13:24.922173 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:24.922173 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:24.922183 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:24.922227 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:24.922504 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:24.923145 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:24.923874 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:24.924183 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:24.924260 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:24.926761 (+ 2501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.130:43749: BlockingRecv error: recv got EOF from 127.28.143.130:43749 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 11 05:13:24 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.130@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:24 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.130@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:24.931808 13457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:24.927045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.130:51695)
0811 05:13:24.927090 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:24.927101 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:24.927142 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:24.927270 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:24.927270 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:24.927425 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:24.927468 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:24.927470 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:24.928159 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:24.928160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:24.928953 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:24.928957 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:24.929308 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:24.929309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:24.929368 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:24.929800 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:24.929806 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:24.931728 (+ 1922us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260811 05:13:24.931912 13218 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:24.927104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:51695 (local address 127.28.143.190:39667)
0811 05:13:24.927143 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:24.927145 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:24.927154 (+ 9us) server_negotiation.cc:407] Connection header received
0811 05:13:24.927194 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:24.927195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:24.927203 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:24.927250 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:24.927565 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:24.928080 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:24.929052 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:24.929238 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:24.929289 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:24.931847 (+ 2558us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.130:51695: BlockingRecv error: recv got EOF from 127.28.143.130:51695 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260811 05:13:24.931960 13455 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 05:13:25.011116 13461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:25.011366 13461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:25.011392 13461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:25.012804 13461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:25.012872 13461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:25.012904 13461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.131
I20260811 05:13:25.014690 13461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:40471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/krb5kdc/oryx_127.28.143.131.keytab
--principal=oryx/127.28.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.131:35903
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.143.131
--webserver_port=35547
--tserver_master_addrs=127.28.143.190:39667
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.131
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:25.015012 13461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:25.015261 13461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:25.020956 13469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:25.020958 13473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:25.020952 13470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:25.021457 13461 server_base.cc:1034] running on GCE node
I20260811 05:13:25.021723 13461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:25.021989 13461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:25.023186 13461 hybrid_clock.cc:648] HybridClock initialized: now 1786425205023166 us; error 31 us; skew 500 ppm
Aug 11 05:13:25 dist-test-slave-f7th krb5kdc[11928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425205, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:25.026165 13461 init.cc:377] Logged in from keytab as oryx/127.28.143.131@KRBTEST.COM (short username oryx)
I20260811 05:13:25.026713 13461 webserver.cc:460] Webserver started at http://127.28.143.131:35547/ using document root <none> and password file <none>
I20260811 05:13:25.027155 13461 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:25.027254 13461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:25.029075 13461 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 05:13:25.029974 13486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:25.030170 13461 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 05:13:25.030278 13461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-2/wal
uuid: "805c4884c1cb41ba8d925e66b9299519"
format_stamp: "Formatted at 2026-08-11 05:13:24 on dist-test-slave-f7th"
server_key: "6b0460b8ee13d5233c2c514356cc5235"
server_key_iv: "34948fc5a6d97498a1e4c4b6f61759b4"
server_key_version: "encryptionkey@0"
I20260811 05:13:25.030683 13461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:25.041708 13461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:25.044230 13461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:25.044407 13461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:25.044728 13461 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:25.045080 13461 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:25.045117 13461 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:25.045147 13461 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:25.045164 13461 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:25.051672 13461 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.131:35903
I20260811 05:13:25.051728 13702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.131:35903 every 8 connection(s)
I20260811 05:13:25.052106 13461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
I20260811 05:13:25.057066 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 13461
Aug 11 05:13:25 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.131@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:25 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.131@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 05:13:25.060547 13218 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:25.052928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:55723 (local address 127.28.143.190:39667)
0811 05:13:25.052975 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:25.052978 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:25.053222 (+ 244us) server_negotiation.cc:407] Connection header received
0811 05:13:25.053711 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:25.053713 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:25.053724 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:25.053773 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:25.054787 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:25.055279 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:25.056367 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:25.056582 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:25.056733 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:25.060482 (+ 3749us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.131:55723: BlockingRecv error: recv got EOF from 127.28.143.131:55723 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 05:13:25.060583 13706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:25.052908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.131:55723)
0811 05:13:25.053080 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:25.053124 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:25.053621 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:25.053839 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:25.053843 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:25.054461 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:25.054697 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:25.054702 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:25.055357 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:25.055358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:25.056253 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:25.056257 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:25.056748 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:25.056750 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:25.056922 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:25.057587 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:25.057600 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:25.060376 (+ 2776us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":79,"threads_started":1}
W20260811 05:13:25.060911 13704 heartbeater.cc:644] Failed to heartbeat to 127.28.143.190:39667 (0 consecutive failures): Not authorized: Failed to ping master at 127.28.143.190:39667: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Aug 11 05:13:25 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.131@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:25 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.131@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:25.065609 13709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:25.057644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47962 (local address 127.28.143.190:39667)
0811 05:13:25.057757 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:25.057759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:25.057768 (+ 9us) server_negotiation.cc:407] Connection header received
0811 05:13:25.057829 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:25.057829 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:25.057840 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:25.057896 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:25.058647 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:25.059304 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:25.060361 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:25.060690 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:25.060755 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:25.062270 (+ 1515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:25.062275 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:25.062275 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:25.062288 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:25.064038 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:25.064404 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:25.064408 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:25.064413 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:25.064515 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:25.064776 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:25.064777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:25.064777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:25.065091 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:25.065207 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:25.065385 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 05:13:25.065498 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":53,"threads_started":1}
I20260811 05:13:25.065642 13706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:25.061153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.131:33075)
0811 05:13:25.061201 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:25.061212 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:25.061255 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:25.061378 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:25.061379 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:25.061509 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:25.061563 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:25.061565 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:25.062410 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:25.062412 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:25.062921 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:25.062923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:25.063259 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:25.063260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:25.063322 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:25.063666 (+ 344us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:25.063670 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:25.065581 (+ 1911us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260811 05:13:25.065702 13218 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:25.061221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:33075 (local address 127.28.143.190:39667)
0811 05:13:25.061274 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:25.061275 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:25.061294 (+ 19us) server_negotiation.cc:407] Connection header received
0811 05:13:25.061311 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:25.061311 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:25.061319 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:25.061355 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:25.061635 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:25.062333 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:25.063002 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:25.063197 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:25.063251 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:25.065644 (+ 2393us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.131:33075: BlockingRecv error: recv got EOF from 127.28.143.131:33075 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 11 05:13:25 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.131@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:25 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.131@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:25.070727 13706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:25.066101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.131:60277)
0811 05:13:25.066141 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:25.066153 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:25.066192 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:25.066406 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:25.066407 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:25.066553 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:25.066595 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:25.066597 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:25.067237 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:25.067239 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:25.067748 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:25.067750 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:25.068112 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:25.068112 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:25.068148 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:25.068572 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:25.068576 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:25.070668 (+ 2092us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":19}
W20260811 05:13:25.070879 13709 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:25.066227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:60277 (local address 127.28.143.190:39667)
0811 05:13:25.066284 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:25.066286 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:25.066294 (+ 8us) server_negotiation.cc:407] Connection header received
0811 05:13:25.066313 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:25.066314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:25.066325 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:25.066394 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:25.066661 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:25.067149 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:25.067840 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:25.068034 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:25.068110 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:25.070816 (+ 2706us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.131:60277: BlockingRecv error: recv got EOF from 127.28.143.131:60277 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260811 05:13:25.070919 13704 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 05:13:25 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.129@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:25 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.129@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:25.798110 13710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:25.793198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.129:42153)
0811 05:13:25.793361 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:25.793373 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:25.793431 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:25.793545 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:25.793546 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:25.793713 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:25.793764 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:25.793766 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:25.794504 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:25.794505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:25.795075 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:25.795077 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:25.795456 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:25.795458 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:25.795512 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:25.796021 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:25.796026 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:25.798038 (+ 2012us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":71,"threads_started":1}
W20260811 05:13:25.798225 13712 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:25.793306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:42153 (local address 127.28.143.190:39667)
0811 05:13:25.793428 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:25.793429 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:25.793439 (+ 10us) server_negotiation.cc:407] Connection header received
0811 05:13:25.793480 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:25.793481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:25.793491 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:25.793539 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:25.793862 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:25.794387 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:25.795155 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:25.795353 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:25.795419 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:25.798155 (+ 2736us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.129:42153: BlockingRecv error: recv got EOF from 127.28.143.129:42153 (error 108)
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":63,"threads_started":1}
Aug 11 05:13:25 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.130@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:25 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.130@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:25.937323 13713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:25.932424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.130:59753)
0811 05:13:25.932583 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:25.932595 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:25.932688 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:25.932817 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:25.932818 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:25.932981 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:25.933039 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:25.933040 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:25.933736 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:25.933738 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:25.934313 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:25.934316 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:25.934706 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:25.934707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:25.934764 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:25.935270 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:25.935275 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:25.937248 (+ 1973us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":77,"threads_started":1}
W20260811 05:13:25.937402 13712 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:25.932528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:59753 (local address 127.28.143.190:39667)
0811 05:13:25.932587 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:25.932589 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:25.932696 (+ 107us) server_negotiation.cc:407] Connection header received
0811 05:13:25.932731 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:25.932732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:25.932742 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:25.932813 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:25.933095 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:25.933624 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:25.934406 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:25.934613 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:25.934675 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:25.937345 (+ 2670us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.130:59753: BlockingRecv error: recv got EOF from 127.28.143.130:59753 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 11 05:13:26 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.131@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:26 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.131@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:26.076429 13714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:26.071326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.131:54807)
0811 05:13:26.071498 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:26.071512 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:26.071572 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:26.071710 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:26.071711 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:26.071901 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:26.071957 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:26.071959 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:26.072716 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:26.072717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:26.073289 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:26.073292 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:26.073642 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:26.073643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:26.073687 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:26.074209 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:26.074215 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:26.076332 (+ 2117us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":78,"threads_started":1}
W20260811 05:13:26.076537 13712 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:26.071476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:54807 (local address 127.28.143.190:39667)
0811 05:13:26.071576 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:26.071578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:26.071620 (+ 42us) server_negotiation.cc:407] Connection header received
0811 05:13:26.071643 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:26.071644 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:26.071656 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:26.071704 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:26.072092 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:26.072593 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:26.073377 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:26.073574 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:26.073633 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:26.076479 (+ 2846us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.131:54807: BlockingRecv error: recv got EOF from 127.28.143.131:54807 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 11 05:13:26 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.129@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:26 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.129@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:26.805500 13716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:26.798960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.129:33979)
0811 05:13:26.799182 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:26.799198 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:26.799271 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:26.799530 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:26.799532 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:26.799723 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:26.799805 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:26.799807 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:26.800866 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:26.800868 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:26.801496 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:26.801499 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:26.802127 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:26.802129 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:26.802225 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:26.802935 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:26.802941 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:26.805418 (+ 2477us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":84,"threads_started":1}
W20260811 05:13:26.805704 13717 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:26.799115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:33979 (local address 127.28.143.190:39667)
0811 05:13:26.799293 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:26.799295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:26.799308 (+ 13us) server_negotiation.cc:407] Connection header received
0811 05:13:26.799378 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:26.799379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:26.799393 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:26.799485 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:26.800009 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:26.800708 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:26.801629 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:26.801878 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:26.802125 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:26.805624 (+ 3499us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.129:33979: BlockingRecv error: recv got EOF from 127.28.143.129:33979 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":54,"threads_started":1}
Aug 11 05:13:26 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.130@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:26 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.130@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:26.943598 13719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:26.937889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.130:55375)
0811 05:13:26.938205 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:26.938224 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:26.938309 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:26.938510 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:26.938511 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:26.938692 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:26.938743 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:26.938745 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:26.939628 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:26.939629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:26.940232 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:26.940236 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:26.940707 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:26.940708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:26.940772 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:26.941229 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:26.941234 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:26.943489 (+ 2255us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":81,"threads_started":1}
W20260811 05:13:26.943951 13717 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:26.938110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:55375 (local address 127.28.143.190:39667)
0811 05:13:26.938252 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:26.938254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:26.938332 (+ 78us) server_negotiation.cc:407] Connection header received
0811 05:13:26.938380 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:26.938381 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:26.938395 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:26.938485 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:26.938850 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:26.939526 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:26.940322 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:26.940582 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:26.940708 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:26.943888 (+ 3180us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.130:55375: BlockingRecv error: recv got EOF from 127.28.143.130:55375 (error 108)
Metrics: {"server-negotiator.queue_time_us":110}
Aug 11 05:13:27 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.131@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:27 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.131@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:27.081640 13721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:27.076966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.131:43563)
0811 05:13:27.077093 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:27.077107 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:27.077166 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:27.077293 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:27.077294 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:27.077451 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:27.077498 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:27.077500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:27.078162 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:27.078163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:27.078705 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:27.078707 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:27.079068 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:27.079069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:27.079108 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:27.079586 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:27.079591 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:27.081574 (+ 1983us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":65,"threads_started":1}
W20260811 05:13:27.081813 13717 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:27.077085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:43563 (local address 127.28.143.190:39667)
0811 05:13:27.077141 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:27.077144 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:27.077153 (+ 9us) server_negotiation.cc:407] Connection header received
0811 05:13:27.077219 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:27.077219 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:27.077232 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:27.077275 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:27.077561 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:27.078081 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:27.078793 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:27.079002 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:27.079060 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:27.081745 (+ 2685us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.131:43563: BlockingRecv error: recv got EOF from 127.28.143.131:43563 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 11 05:13:27 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.129@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:27 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.129@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:27.813582 13722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:27.806295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.129:59061)
0811 05:13:27.806529 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:27.806545 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:27.806629 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:27.807077 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:27.807078 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:27.807299 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:27.807357 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:27.807360 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:27.808223 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:27.808224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:27.809135 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:27.809141 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:27.809997 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:27.809998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:27.810068 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:27.810962 (+ 894us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:27.810969 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:27.813488 (+ 2519us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":111,"threads_started":1}
W20260811 05:13:27.813745 13725 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:27.806443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:59061 (local address 127.28.143.190:39667)
0811 05:13:27.806628 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:27.806630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:27.806714 (+ 84us) server_negotiation.cc:407] Connection header received
0811 05:13:27.806754 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:27.806755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:27.806771 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:27.806869 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:27.807499 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:27.808049 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:27.809510 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:27.809801 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:27.809981 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:27.813674 (+ 3693us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.129:59061: BlockingRecv error: recv got EOF from 127.28.143.129:59061 (error 108)
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":95,"threads_started":1}
Aug 11 05:13:27 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.130@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:27 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.130@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:27.950564 13726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:27.944152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.130:35073)
0811 05:13:27.944310 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:27.944326 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:27.944407 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:27.944557 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:27.944558 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:27.944761 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:27.944815 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:27.944817 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:27.945832 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:27.945833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:27.946597 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:27.946602 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:27.947384 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:27.947385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:27.947496 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:27.948249 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:27.948256 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:27.950499 (+ 2243us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":66,"threads_started":1}
W20260811 05:13:27.950726 13725 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:27.944277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:35073 (local address 127.28.143.190:39667)
0811 05:13:27.944410 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:27.944412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:27.944444 (+ 32us) server_negotiation.cc:407] Connection header received
0811 05:13:27.944481 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:27.944482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:27.944494 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:27.944537 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:27.944946 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:27.945735 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:27.946831 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:27.947252 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:27.947336 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:27.950659 (+ 3323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.130:35073: BlockingRecv error: recv got EOF from 127.28.143.130:35073 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
Aug 11 05:13:28 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.131@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:28 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.131@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:28.087641 13727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:28.082316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.131:57967)
0811 05:13:28.082485 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:28.082498 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:28.082581 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:28.082872 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:28.082874 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:28.083100 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:28.083149 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:28.083151 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:28.083807 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:28.083809 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:28.084382 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:28.084384 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:28.084757 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:28.084758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:28.084814 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:28.085297 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:28.085301 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:28.087568 (+ 2267us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":77,"threads_started":1}
W20260811 05:13:28.087826 13725 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:28.082482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:57967 (local address 127.28.143.190:39667)
0811 05:13:28.082548 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:28.082551 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:28.082561 (+ 10us) server_negotiation.cc:407] Connection header received
0811 05:13:28.082660 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:28.082661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:28.082675 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:28.082855 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:28.083223 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:28.083730 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:28.084464 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:28.084697 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:28.084757 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:28.087749 (+ 2992us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.131:57967: BlockingRecv error: recv got EOF from 127.28.143.131:57967 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 11 05:13:28 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.129@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:28 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.129@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:28.819446 13729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:28.814317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.129:41131)
0811 05:13:28.814466 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:28.814478 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:28.814538 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:28.814798 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:28.814800 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:28.815003 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:28.815056 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:28.815058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:28.815769 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:28.815770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:28.816315 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:28.816317 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:28.816735 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:28.816736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:28.816787 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:28.817338 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:28.817344 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:28.819380 (+ 2036us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":98,"threads_started":1}
W20260811 05:13:28.819635 13730 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:28.814486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:41131 (local address 127.28.143.190:39667)
0811 05:13:28.814618 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:28.814620 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:28.814629 (+ 9us) server_negotiation.cc:407] Connection header received
0811 05:13:28.814657 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:28.814657 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:28.814668 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:28.814747 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:28.815150 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:28.815662 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:28.816419 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:28.816652 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:28.816706 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:28.819573 (+ 2867us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.129:41131: BlockingRecv error: recv got EOF from 127.28.143.129:41131 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":64,"threads_started":1}
Aug 11 05:13:28 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.130@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:28 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.130@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:28.960500 13732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:28.951121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.130:36903)
0811 05:13:28.951350 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:28.951363 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:28.951451 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:28.951629 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:28.951630 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:28.951845 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:28.951906 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:28.951908 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:28.952813 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:28.952815 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:28.953376 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:28.953378 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:28.953736 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:28.953737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:28.953791 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:28.958331 (+ 4540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:28.958339 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:28.960411 (+ 2072us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":73,"threads_started":1}
W20260811 05:13:28.960683 13730 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:28.951259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:36903 (local address 127.28.143.190:39667)
0811 05:13:28.951319 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:28.951321 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:28.951439 (+ 118us) server_negotiation.cc:407] Connection header received
0811 05:13:28.951520 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:28.951521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:28.951536 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:28.951602 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:28.952002 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:28.952726 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:28.953440 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:28.953637 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:28.953699 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:28.960589 (+ 6890us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.130:36903: BlockingRecv error: recv got EOF from 127.28.143.130:36903 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 11 05:13:29 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.131@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:29 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.131@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:29.093472 13733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:29.088206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.131:55183)
0811 05:13:29.088329 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:29.088375 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:29.088431 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:29.088573 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:29.088574 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:29.088768 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:29.088819 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:29.088821 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:29.089651 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:29.089653 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:29.090203 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:29.090206 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:29.090540 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:29.090541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:29.090605 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:29.091168 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:29.091173 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:29.093402 (+ 2229us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":60,"threads_started":1}
W20260811 05:13:29.093600 13730 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:29.088307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:55183 (local address 127.28.143.190:39667)
0811 05:13:29.088409 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:29.088412 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:29.088421 (+ 9us) server_negotiation.cc:407] Connection header received
0811 05:13:29.088504 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:29.088505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:29.088518 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:29.088566 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:29.088952 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:29.089542 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:29.090290 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:29.090475 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:29.090531 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:29.093533 (+ 3002us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.131:55183: BlockingRecv error: recv got EOF from 127.28.143.131:55183 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 11 05:13:29 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.129@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:29 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.129@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:29.825673 13736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:29.820206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.129:49157)
0811 05:13:29.820400 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:29.820412 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:29.820481 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:29.820721 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:29.820722 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:29.820968 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:29.821020 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:29.821022 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:29.821794 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:29.821795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:29.822354 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:29.822356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:29.822836 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:29.822837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:29.822904 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:29.823481 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:29.823487 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:29.825587 (+ 2100us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":79,"threads_started":1}
W20260811 05:13:29.825790 13738 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:29.820252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:49157 (local address 127.28.143.190:39667)
0811 05:13:29.820404 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:29.820406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:29.820519 (+ 113us) server_negotiation.cc:407] Connection header received
0811 05:13:29.820558 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:29.820559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:29.820573 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:29.820684 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:29.821149 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:29.821676 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:29.822487 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:29.822726 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:29.822791 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:29.825718 (+ 2927us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.129:49157: BlockingRecv error: recv got EOF from 127.28.143.129:49157 (error 108)
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":105,"threads_started":1}
Aug 11 05:13:29 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.130@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:29 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.130@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:29.967801 13739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:29.961242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.130:58855)
0811 05:13:29.961465 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:29.961490 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:29.961640 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:29.961826 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:29.961827 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:29.962029 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:29.962114 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:29.962115 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:29.962846 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:29.962847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:29.963412 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:29.963415 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:29.964050 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:29.964051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:29.964129 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:29.964986 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:29.964994 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:29.967715 (+ 2721us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":97,"threads_started":1}
W20260811 05:13:29.968020 13738 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:29.961506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:58855 (local address 127.28.143.190:39667)
0811 05:13:29.961622 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:29.961624 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:29.961637 (+ 13us) server_negotiation.cc:407] Connection header received
0811 05:13:29.961724 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:29.961725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:29.961738 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:29.961783 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:29.962216 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:29.962737 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:29.963608 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:29.963897 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:29.964008 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:29.967953 (+ 3945us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.130:58855: BlockingRecv error: recv got EOF from 127.28.143.130:58855 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 11 05:13:30 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.131@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:30 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.131@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:30.100059 13740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:30.094099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.131:40799)
0811 05:13:30.094300 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:30.094311 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:30.094375 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:30.094541 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:30.094543 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:30.094715 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:30.094765 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:30.094767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:30.095520 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:30.095523 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:30.096171 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:30.096175 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:30.096908 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:30.096909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:30.096961 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:30.097544 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:30.097597 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:30.099993 (+ 2396us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":102,"threads_started":1}
W20260811 05:13:30.100186 13738 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:30.094213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:40799 (local address 127.28.143.190:39667)
0811 05:13:30.094288 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:30.094290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:30.094377 (+ 87us) server_negotiation.cc:407] Connection header received
0811 05:13:30.094425 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:30.094425 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:30.094438 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:30.094488 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:30.094873 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:30.095413 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:30.096302 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:30.096722 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:30.096845 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:30.100120 (+ 3275us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.131:40799: BlockingRecv error: recv got EOF from 127.28.143.131:40799 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 11 05:13:30 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.129@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:30 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.129@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:30.831350 13741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:30.826340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.129:57221)
0811 05:13:30.826469 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:30.826481 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:30.826539 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:30.826740 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:30.826741 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:30.826928 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:30.826980 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:30.826982 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:30.827686 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:30.827687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:30.828355 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:30.828359 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:30.828866 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:30.828867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:30.828954 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:30.829507 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:30.829512 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:30.831284 (+ 1772us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":70,"threads_started":1}
W20260811 05:13:30.831537 13742 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:30.826488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:57221 (local address 127.28.143.190:39667)
0811 05:13:30.826626 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:30.826628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:30.826636 (+ 8us) server_negotiation.cc:407] Connection header received
0811 05:13:30.826659 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:30.826659 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:30.826671 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:30.826730 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:30.827075 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:30.827608 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:30.828471 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:30.828709 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:30.828774 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:30.831460 (+ 2686us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.129:57221: BlockingRecv error: recv got EOF from 127.28.143.129:57221 (error 108)
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":65,"threads_started":1}
Aug 11 05:13:30 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.130@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:30 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.130@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:30.975144 13745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:30.968464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.130:59735)
0811 05:13:30.968658 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:30.968673 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:30.968746 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:30.968874 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:30.968875 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:30.969048 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:30.969117 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:30.969119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:30.969872 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:30.969874 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:30.970436 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:30.970439 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:30.970934 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:30.970935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:30.970989 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:30.971567 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:30.971573 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:30.975066 (+ 3493us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":66,"threads_started":1}
W20260811 05:13:30.975314 13742 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:30.968611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:59735 (local address 127.28.143.190:39667)
0811 05:13:30.968719 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:30.968721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:30.968736 (+ 15us) server_negotiation.cc:407] Connection header received
0811 05:13:30.968802 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:30.968803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:30.968815 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:30.968868 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:30.969209 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:30.969719 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:30.970559 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:30.970760 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:30.970832 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:30.975249 (+ 4417us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.130:59735: BlockingRecv error: recv got EOF from 127.28.143.130:59735 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 11 05:13:31 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.131@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:31 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.131@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:31.107236 13747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:31.100737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.131:56085)
0811 05:13:31.102370 (+ 1633us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:31.102386 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:31.102470 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:31.102691 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:31.102693 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:31.102887 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:31.102938 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:31.102940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:31.103663 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:31.103664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:31.104202 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:31.104204 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:31.104597 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:31.104599 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:31.104679 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:31.105266 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:31.105272 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:31.107175 (+ 1903us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1607,"thread_start_us":1542,"threads_started":1}
W20260811 05:13:31.107365 13742 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:31.100824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:56085 (local address 127.28.143.190:39667)
0811 05:13:31.100873 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:31.100876 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:31.102491 (+ 1615us) server_negotiation.cc:407] Connection header received
0811 05:13:31.102525 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:31.102526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:31.102541 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:31.102608 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:31.103059 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:31.103569 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:31.104299 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:31.104500 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:31.104557 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:31.107297 (+ 2740us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.131:56085: BlockingRecv error: recv got EOF from 127.28.143.131:56085 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 11 05:13:31 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.129@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:31 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.129@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:31.838413 13748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:31.832055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.129:44783)
0811 05:13:31.832197 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:31.832210 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:31.832264 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:31.832506 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:31.832507 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:31.832797 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:31.832906 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:31.832909 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:31.833830 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:31.833831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:31.834455 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:31.834458 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:31.834869 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:31.834870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:31.834928 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:31.835492 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:31.835498 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:31.838319 (+ 2821us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":77,"threads_started":1}
W20260811 05:13:31.838641 13750 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:31.832226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:44783 (local address 127.28.143.190:39667)
0811 05:13:31.832380 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:31.832383 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:31.832390 (+ 7us) server_negotiation.cc:407] Connection header received
0811 05:13:31.832412 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:31.832412 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:31.832423 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:31.832479 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:31.833024 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:31.833736 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:31.834539 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:31.834757 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:31.834816 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:31.838561 (+ 3745us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.129:44783: BlockingRecv error: recv got EOF from 127.28.143.129:44783 (error 108)
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":70,"threads_started":1}
Aug 11 05:13:31 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.130@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:31 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.130@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:31.981060 13752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:31.975828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.130:54421)
0811 05:13:31.975996 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:31.976008 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:31.976068 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:31.976198 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:31.976199 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:31.976373 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:31.976426 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:31.976428 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:31.977215 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:31.977216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:31.977764 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:31.977766 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:31.978165 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:31.978166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:31.978209 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:31.978692 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:31.978696 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:31.980982 (+ 2286us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":100,"threads_started":1}
W20260811 05:13:31.981105 13750 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:31.975939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:54421 (local address 127.28.143.190:39667)
0811 05:13:31.976046 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:31.976047 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:31.976087 (+ 40us) server_negotiation.cc:407] Connection header received
0811 05:13:31.976121 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:31.976122 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:31.976134 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:31.976182 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:31.976508 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:31.977099 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:31.977877 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:31.978093 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:31.978157 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:31.981057 (+ 2900us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.130:54421: BlockingRecv error: recv got EOF from 127.28.143.130:54421 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 11 05:13:32 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.131@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:32 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.131@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:32.113252 13756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:32.107797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.131:59803)
0811 05:13:32.107963 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:32.107974 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:32.108061 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:32.108250 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:32.108251 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:32.108425 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:32.108495 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:32.108497 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:32.109254 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:32.109255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:32.109953 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:32.109957 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:32.110381 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:32.110382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:32.110433 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:32.110987 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:32.110993 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:32.113170 (+ 2177us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":74,"threads_started":1}
W20260811 05:13:32.113722 13750 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:32.107974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:59803 (local address 127.28.143.190:39667)
0811 05:13:32.108034 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:32.108035 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:32.108048 (+ 13us) server_negotiation.cc:407] Connection header received
0811 05:13:32.108129 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:32.108130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:32.108143 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:32.108187 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:32.108598 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:32.109155 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:32.110107 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:32.110297 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:32.110362 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:32.113300 (+ 2938us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.131:59803: BlockingRecv error: recv got EOF from 127.28.143.131:59803 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 11 05:13:32 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.129@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:32 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.129@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:32.844542 13758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:32.839055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.129:59527)
0811 05:13:32.839233 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:32.839246 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:32.839307 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:32.839598 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:32.839599 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:32.839779 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:32.839831 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:32.839833 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:32.840679 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:32.840680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:32.841267 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:32.841270 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:32.841693 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:32.841694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:32.841748 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:32.842386 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:32.842391 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:32.844447 (+ 2056us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":66,"threads_started":1}
W20260811 05:13:32.844763 13759 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:32.839284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:59527 (local address 127.28.143.190:39667)
0811 05:13:32.839437 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:32.839438 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:32.839447 (+ 9us) server_negotiation.cc:407] Connection header received
0811 05:13:32.839470 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:32.839471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:32.839481 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:32.839571 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:32.839962 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:32.840569 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:32.841373 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:32.841586 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:32.841656 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:32.844688 (+ 3032us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.129:59527: BlockingRecv error: recv got EOF from 127.28.143.129:59527 (error 108)
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":62,"threads_started":1}
Aug 11 05:13:32 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.130@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:32 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.130@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:32.987705 13760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:32.981693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.130:51297)
0811 05:13:32.981834 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:32.981846 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:32.981946 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:32.982147 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:32.982149 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:32.982323 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:32.982380 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:32.982382 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:32.983312 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:32.983314 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:32.984031 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:32.984035 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:32.984464 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:32.984465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:32.984575 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:32.985362 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:32.985367 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:32.987615 (+ 2248us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":73,"threads_started":1}
W20260811 05:13:32.987986 13759 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:32.981828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:51297 (local address 127.28.143.190:39667)
0811 05:13:32.981923 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:32.981925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:32.981940 (+ 15us) server_negotiation.cc:407] Connection header received
0811 05:13:32.982000 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:32.982001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:32.982013 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:32.982076 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:32.982499 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:32.983153 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:32.984154 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:32.984355 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:32.984420 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:32.987905 (+ 3485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.130:51297: BlockingRecv error: recv got EOF from 127.28.143.130:51297 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 11 05:13:33 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.131@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:33 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.131@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:33.119796 13763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:33.114408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.131:38259)
0811 05:13:33.114612 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:33.114626 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:33.114683 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:33.115156 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:33.115157 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:33.115350 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:33.115400 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:33.115402 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:33.116205 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:33.116207 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:33.116890 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:33.116895 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:33.117317 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:33.117318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:33.117364 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:33.117884 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:33.117889 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:33.119732 (+ 1843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":101,"threads_started":1}
W20260811 05:13:33.119961 13759 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:33.114577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:38259 (local address 127.28.143.190:39667)
0811 05:13:33.114669 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:33.114671 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:33.114682 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:13:33.114734 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:33.114735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:33.114748 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:33.115035 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:33.115554 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:33.116092 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:33.117009 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:33.117236 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:33.117298 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:33.119889 (+ 2591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.131:38259: BlockingRecv error: recv got EOF from 127.28.143.131:38259 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 11 05:13:33 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.129@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:33 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.129@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:33.850703 13765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:33.845139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.129:42801)
0811 05:13:33.845296 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:33.845307 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:33.845371 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:33.845646 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:33.845647 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:33.845827 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:33.845878 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:33.845880 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:33.846572 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:33.846573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:33.847129 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:33.847131 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:33.847513 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:33.847514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:33.847567 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:33.848114 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:33.848119 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:33.850637 (+ 2518us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":72,"threads_started":1}
W20260811 05:13:33.850891 13766 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:33.845269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:42801 (local address 127.28.143.190:39667)
0811 05:13:33.845392 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:33.845394 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:33.845404 (+ 10us) server_negotiation.cc:407] Connection header received
0811 05:13:33.845514 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:33.845514 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:33.845526 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:33.845589 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:33.845965 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:33.846474 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:33.847221 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:33.847418 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:33.847473 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:33.850816 (+ 3343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.129:42801: BlockingRecv error: recv got EOF from 127.28.143.129:42801 (error 108)
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":52,"threads_started":1}
Aug 11 05:13:33 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.130@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:33 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.130@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:33.997009 13767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:33.988420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.130:46243)
0811 05:13:33.988590 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:33.988602 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:33.988695 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:33.988828 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:33.988829 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:33.989027 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:33.989076 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:33.989078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:33.989805 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:33.989806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:33.990503 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:33.990506 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:33.990863 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:33.990864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:33.990920 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:33.994920 (+ 4000us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:33.994928 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:33.996925 (+ 1997us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":95,"threads_started":1}
W20260811 05:13:33.997133 13766 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:33.988595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:46243 (local address 127.28.143.190:39667)
0811 05:13:33.988695 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:33.988697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:33.988708 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:13:33.988750 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:33.988751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:33.988765 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:33.988815 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:33.989153 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:33.989729 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:33.990579 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:33.990794 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:33.990852 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:33.997062 (+ 6210us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.130:46243: BlockingRecv error: recv got EOF from 127.28.143.130:46243 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 11 05:13:34 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.131@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:34 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.131@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:34.125536 13771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:34.120330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.131:52827)
0811 05:13:34.120507 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:34.120523 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:34.120597 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:34.120964 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:34.120966 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:34.121142 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:34.121190 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:34.121192 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:34.121909 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:34.121910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:34.122494 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:34.122498 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:34.122903 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:34.122904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:34.122957 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:34.123416 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:34.123421 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:34.125462 (+ 2041us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":78,"threads_started":1}
W20260811 05:13:34.125631 13766 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:34.120543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:52827 (local address 127.28.143.190:39667)
0811 05:13:34.120685 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:34.120687 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:34.120706 (+ 19us) server_negotiation.cc:407] Connection header received
0811 05:13:34.120826 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:34.120827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:34.120846 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:34.120914 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:34.121291 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:34.121793 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:34.122589 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:34.122775 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:34.122834 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:34.125566 (+ 2732us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.131:52827: BlockingRecv error: recv got EOF from 127.28.143.131:52827 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
Aug 11 05:13:34 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.129@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:34 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.129@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:34.857218 13773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:34.851379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.129:43107)
0811 05:13:34.851542 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:34.851553 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:34.851608 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:34.851932 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:34.851934 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:34.852134 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:34.852189 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:34.852192 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:34.853142 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:34.853143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:34.853709 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:34.853711 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:34.854240 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:34.854242 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:34.854294 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:34.854808 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:34.854813 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:34.857150 (+ 2337us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":92,"threads_started":1}
W20260811 05:13:34.857409 13774 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:34.851490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:43107 (local address 127.28.143.190:39667)
0811 05:13:34.851701 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:34.851702 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:34.851713 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:13:34.851741 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:34.851742 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:34.851753 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:34.851823 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:34.852313 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:34.852923 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:34.853839 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:34.854045 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:34.854134 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:34.857322 (+ 3188us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.129:43107: BlockingRecv error: recv got EOF from 127.28.143.129:43107 (error 108)
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":80,"threads_started":1}
Aug 11 05:13:34 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.130@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:34 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.130@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:35.002279 13776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:34.997567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.130:44385)
0811 05:13:34.997709 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:34.997723 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:34.997786 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:34.997921 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:34.997922 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:34.998115 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:34.998163 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:34.998165 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:34.998824 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:34.998825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:34.999392 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:34.999394 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:34.999725 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:34.999725 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:34.999781 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:35.000248 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:35.000254 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:35.002215 (+ 1961us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":65,"threads_started":1}
W20260811 05:13:35.002426 13774 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:34.997694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:44385 (local address 127.28.143.190:39667)
0811 05:13:34.997759 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:34.997761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:34.997771 (+ 10us) server_negotiation.cc:407] Connection header received
0811 05:13:34.997850 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:34.997851 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:34.997864 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:34.997904 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:34.998235 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:34.998741 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:34.999474 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:34.999661 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:34.999715 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:35.002357 (+ 2642us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.130:44385: BlockingRecv error: recv got EOF from 127.28.143.130:44385 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 11 05:13:35 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.131@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:35 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.131@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:35.131642 13777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:35.126334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.131:34309)
0811 05:13:35.126585 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:35.126601 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:35.126667 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:35.126817 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:35.126818 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:35.127047 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:35.127098 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:35.127100 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:35.127843 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:35.127844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:35.128427 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:35.128430 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:35.128845 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:35.128846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:35.128971 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:35.129543 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:35.129549 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:35.131558 (+ 2009us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":197,"threads_started":1}
W20260811 05:13:35.131723 13774 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:35.126598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:34309 (local address 127.28.143.190:39667)
0811 05:13:35.126666 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:35.126668 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:35.126702 (+ 34us) server_negotiation.cc:407] Connection header received
0811 05:13:35.126731 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:35.126732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:35.126750 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:35.126792 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:35.127223 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:35.127733 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:35.128536 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:35.128773 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:35.128845 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:35.131667 (+ 2822us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.131:34309: BlockingRecv error: recv got EOF from 127.28.143.131:34309 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 11 05:13:35 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.129@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:35 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.129@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:35.863574 13778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:35.858049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.129:41049)
0811 05:13:35.858226 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:35.858239 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:35.858307 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:35.858524 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:35.858525 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:35.858711 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:35.858761 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:35.858763 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:35.859537 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:35.859538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:35.860153 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:35.860156 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:35.860655 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:35.860657 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:35.860742 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:35.861423 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:35.861429 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:35.863482 (+ 2053us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":78,"threads_started":1}
W20260811 05:13:35.863770 13779 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:35.858207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:41049 (local address 127.28.143.190:39667)
0811 05:13:35.858339 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:35.858342 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:35.858353 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:13:35.858385 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:35.858386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:35.858400 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:35.858474 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:35.858842 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:35.859401 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:35.860261 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:35.860471 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:35.860547 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:35.863690 (+ 3143us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.129:41049: BlockingRecv error: recv got EOF from 127.28.143.129:41049 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":60,"threads_started":1}
Aug 11 05:13:36 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.130@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:36 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.130@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:36.008955 13781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:36.002872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.130:54891)
0811 05:13:36.003117 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:36.003131 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:36.003212 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:36.003386 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:36.003388 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:36.003578 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:36.003626 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:36.003628 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:36.004382 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:36.004383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:36.005241 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:36.005244 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:36.005785 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:36.005786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:36.005841 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:36.006454 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:36.006459 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:36.008870 (+ 2411us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":81,"threads_started":1}
W20260811 05:13:36.009173 13779 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:36.003005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:54891 (local address 127.28.143.190:39667)
0811 05:13:36.003102 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:36.003105 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:36.003212 (+ 107us) server_negotiation.cc:407] Connection header received
0811 05:13:36.003266 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:36.003267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:36.003281 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:36.003337 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:36.003735 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:36.004266 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:36.005372 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:36.005654 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:36.005720 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:36.009117 (+ 3397us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.130:54891: BlockingRecv error: recv got EOF from 127.28.143.130:54891 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 11 05:13:36 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.131@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:36 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.131@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:36.137590 13782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:36.132303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.131:46141)
0811 05:13:36.132494 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:36.132513 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:36.132588 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:36.132788 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:36.132790 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:36.133047 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:36.133109 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:36.133112 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:36.133827 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:36.133829 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:36.134422 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:36.134425 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:36.134765 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:36.134766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:36.134824 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:36.135518 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:36.135524 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:36.137480 (+ 1956us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":88,"threads_started":1}
W20260811 05:13:36.137679 13779 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:36.132453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:46141 (local address 127.28.143.190:39667)
0811 05:13:36.132553 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:36.132555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:36.132572 (+ 17us) server_negotiation.cc:407] Connection header received
0811 05:13:36.132703 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:36.132704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:36.132720 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:36.132768 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:36.133188 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:36.133737 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:36.134509 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:36.134695 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:36.134753 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:36.137620 (+ 2867us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.131:46141: BlockingRecv error: recv got EOF from 127.28.143.131:46141 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 11 05:13:36 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.129@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:36 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.129@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:36.869751 13784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:36.864217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.129:33267)
0811 05:13:36.864443 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:36.864459 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:36.864527 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:36.864922 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:36.864924 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:36.865117 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:36.865166 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:36.865168 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:36.865904 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:36.865905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:36.866461 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:36.866463 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:36.866895 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:36.866896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:36.866956 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:36.867560 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:36.867566 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:36.869670 (+ 2104us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":160,"threads_started":1}
W20260811 05:13:36.869858 13786 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:36.864379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:33267 (local address 127.28.143.190:39667)
0811 05:13:36.864641 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:36.864643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:36.864653 (+ 10us) server_negotiation.cc:407] Connection header received
0811 05:13:36.864683 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:36.864684 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:36.864754 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:36.864847 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:36.865269 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:36.865793 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:36.866570 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:36.866782 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:36.866864 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:36.869790 (+ 2926us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.129:33267: BlockingRecv error: recv got EOF from 127.28.143.129:33267 (error 108)
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":155,"threads_started":1}
Aug 11 05:13:37 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.130@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:37 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.130@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 05:13:37.016981 13786 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:37.009718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:39769 (local address 127.28.143.190:39667)
0811 05:13:37.009939 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:37.009942 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:37.009963 (+ 21us) server_negotiation.cc:407] Connection header received
0811 05:13:37.010019 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:37.010021 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:37.010045 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:37.010159 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:37.010864 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:37.011406 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:37.012403 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:37.012640 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:37.012700 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:37.016913 (+ 4213us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.130:39769: BlockingRecv error: recv got EOF from 127.28.143.130:39769 (error 108)
Metrics: {"server-negotiator.queue_time_us":162}
I20260811 05:13:37.017683 13790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:37.009603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.130:39769)
0811 05:13:37.009810 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:37.009824 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:37.009893 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:37.010321 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:37.010323 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:37.010573 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:37.010639 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:37.010642 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:37.011584 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:37.011586 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:37.012186 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:37.012189 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:37.012827 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:37.012828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:37.012886 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:37.013558 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:37.013565 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:37.017271 (+ 3706us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":80,"threads_started":1}
Aug 11 05:13:37 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.131@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:37 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.131@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:37.143534 13791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:37.138241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.131:46973)
0811 05:13:37.138409 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:37.138421 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:37.138485 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:37.138623 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:37.138624 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:37.138831 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:37.138887 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:37.138889 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:37.139762 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:37.139764 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:37.140441 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:37.140445 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:37.140886 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:37.140887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:37.140953 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:37.141420 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:37.141425 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:37.143434 (+ 2009us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":68,"threads_started":1}
W20260811 05:13:37.143616 13786 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:37.138392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:46973 (local address 127.28.143.190:39667)
0811 05:13:37.138458 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:37.138460 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:37.138471 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:13:37.138540 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:37.138541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:37.138555 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:37.138605 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:37.138945 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:37.139665 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:37.140547 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:37.140815 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:37.140879 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:37.143558 (+ 2679us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.131:46973: BlockingRecv error: recv got EOF from 127.28.143.131:46973 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 11 05:13:37 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.129@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:37 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.129@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:37.875679 13793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:37.870460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.129:48541)
0811 05:13:37.870609 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:37.870620 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:37.870672 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:37.870865 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:37.870866 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:37.871092 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:37.871139 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:37.871141 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:37.871852 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:37.871854 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:37.872475 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:37.872477 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:37.872940 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:37.872941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:37.872996 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:37.873596 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:37.873601 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:37.875596 (+ 1995us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":66,"threads_started":1}
W20260811 05:13:37.875789 13795 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:37.870586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:48541 (local address 127.28.143.190:39667)
0811 05:13:37.870722 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:37.870724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:37.870734 (+ 10us) server_negotiation.cc:407] Connection header received
0811 05:13:37.870765 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:37.870766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:37.870777 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:37.870847 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:37.871223 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:37.871749 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:37.872585 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:37.872823 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:37.872886 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:37.875715 (+ 2829us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.129:48541: BlockingRecv error: recv got EOF from 127.28.143.129:48541 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":66,"threads_started":1}
Aug 11 05:13:38 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.130@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:38 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.130@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:38.023584 13798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:38.018351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.130:39865)
0811 05:13:38.018502 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:38.018514 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:38.018590 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:38.018715 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:38.018716 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:38.018887 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:38.018936 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:38.018938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:38.019708 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:38.019708 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:38.020311 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:38.020313 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:38.020751 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:38.020752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:38.020799 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:38.021337 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:38.021342 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:38.023494 (+ 2152us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":78,"threads_started":1}
W20260811 05:13:38.023684 13795 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:38.018466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:39865 (local address 127.28.143.190:39667)
0811 05:13:38.018532 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:38.018534 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:38.018572 (+ 38us) server_negotiation.cc:407] Connection header received
0811 05:13:38.018648 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:38.018648 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:38.018659 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:38.018703 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:38.019046 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:38.019604 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:38.020442 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:38.020671 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:38.020728 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:38.023621 (+ 2893us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.130:39865: BlockingRecv error: recv got EOF from 127.28.143.130:39865 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 11 05:13:38 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.131@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:38 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.131@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:38.149689 13799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:38.144208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.131:46245)
0811 05:13:38.144361 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:38.144376 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:38.144471 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:38.144682 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:38.144684 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:38.144913 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:38.144978 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:38.144981 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:38.146035 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:38.146044 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:38.146583 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:38.146585 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:38.147054 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:38.147055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:38.147111 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:38.147610 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:38.147615 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:38.149608 (+ 1993us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":75,"threads_started":1}
W20260811 05:13:38.149784 13795 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:38.144308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:46245 (local address 127.28.143.190:39667)
0811 05:13:38.144372 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:38.144375 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:38.144463 (+ 88us) server_negotiation.cc:407] Connection header received
0811 05:13:38.144557 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:38.144558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:38.144574 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:38.144667 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:38.145115 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:38.145866 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:38.146699 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:38.146920 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:38.146999 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:38.149728 (+ 2729us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.131:46245: BlockingRecv error: recv got EOF from 127.28.143.131:46245 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 11 05:13:38 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.129@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:38 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.129@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:38.886966 13800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:38.876220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.129:42717)
0811 05:13:38.880723 (+ 4503us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:38.880740 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:38.880848 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:38.881152 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:38.881153 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:38.881378 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:38.881445 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:38.881448 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:38.882401 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:38.882403 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:38.883400 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:38.883405 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:38.883928 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:38.883929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:38.883997 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:38.884500 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:38.884507 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:38.886884 (+ 2377us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":4470,"thread_start_us":71,"threads_started":1}
W20260811 05:13:38.887086 13801 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:38.876403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:42717 (local address 127.28.143.190:39667)
0811 05:13:38.876524 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:38.876527 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:38.880855 (+ 4328us) server_negotiation.cc:407] Connection header received
0811 05:13:38.880954 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:38.880956 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:38.880973 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:38.881109 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:38.881558 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:38.882300 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:38.883542 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:38.883836 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:38.883911 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:38.887019 (+ 3108us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.129:42717: BlockingRecv error: recv got EOF from 127.28.143.129:42717 (error 108)
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":46,"threads_started":1}
Aug 11 05:13:39 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.130@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:39 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.130@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:39.032662 13802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:39.024251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.130:49053)
0811 05:13:39.024387 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:39.024398 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:39.024466 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:39.024745 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:39.024746 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:39.024930 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:39.024989 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:39.024991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:39.025784 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:39.025785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:39.026392 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:39.026396 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:39.026831 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:39.026832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:39.026963 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:39.030559 (+ 3596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:39.030566 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:39.032564 (+ 1998us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":63,"threads_started":1}
W20260811 05:13:39.032796 13801 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:39.024406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:49053 (local address 127.28.143.190:39667)
0811 05:13:39.024519 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:39.024522 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:39.024543 (+ 21us) server_negotiation.cc:407] Connection header received
0811 05:13:39.024574 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:39.024575 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:39.024586 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:39.024649 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:39.025077 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:39.025666 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:39.026497 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:39.026746 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:39.026799 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:39.032730 (+ 5931us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.130:49053: BlockingRecv error: recv got EOF from 127.28.143.130:49053 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 11 05:13:39 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.131@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:39 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.131@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:39.155249 13807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:39.150305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.131:35265)
0811 05:13:39.150505 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:39.150515 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:39.150567 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:39.150707 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:39.150708 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:39.150903 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:39.150973 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:39.150975 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:39.151667 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:39.151668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:39.152253 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:39.152255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:39.152705 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:39.152706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:39.152785 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:39.153250 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:39.153255 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:39.155150 (+ 1895us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":78,"threads_started":1}
W20260811 05:13:39.155351 13801 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:39.150387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:35265 (local address 127.28.143.190:39667)
0811 05:13:39.150436 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:39.150437 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:39.150557 (+ 120us) server_negotiation.cc:407] Connection header received
0811 05:13:39.150612 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:39.150613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:39.150629 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:39.150686 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:39.151050 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:39.151568 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:39.152354 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:39.152563 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:39.152645 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:39.155298 (+ 2653us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.131:35265: BlockingRecv error: recv got EOF from 127.28.143.131:35265 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 11 05:13:39 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.129@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:39 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.129@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:39.892795 13809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:39.887697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.129:37729)
0811 05:13:39.887870 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:39.887882 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:39.887936 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:39.888159 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:39.888160 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:39.888379 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:39.888451 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:39.888454 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:39.889200 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:39.889201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:39.889743 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:39.889745 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:39.890146 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:39.890147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:39.890192 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:39.890709 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:39.890715 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:39.892731 (+ 2016us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":86,"threads_started":1}
W20260811 05:13:39.892974 13810 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:39.887806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:37729 (local address 127.28.143.190:39667)
0811 05:13:39.887951 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:39.887953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:39.888032 (+ 79us) server_negotiation.cc:407] Connection header received
0811 05:13:39.888054 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:39.888055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:39.888066 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:39.888129 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:39.888542 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:39.889083 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:39.889849 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:39.890067 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:39.890131 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:39.892887 (+ 2756us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.129:37729: BlockingRecv error: recv got EOF from 127.28.143.129:37729 (error 108)
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":67,"threads_started":1}
Aug 11 05:13:40 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.130@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:40 dist-test-slave-f7th krb5kdc[11928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.143.130@KRBTEST.COM for oryx/127.28.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:40.038419 13811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:40.033382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39667 (local address 127.28.143.130:43773)
0811 05:13:40.033543 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:40.033558 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:40.033635 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:40.033787 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:40.033788 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:40.033982 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:40.034031 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:40.034033 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:40.034717 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:40.034719 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:40.035263 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:40.035265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:40.035668 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:40.035670 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:40.035727 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:40.036253 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:40.036258 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:40.038330 (+ 2072us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.143.190:39667: Server oryx/127.28.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":75,"threads_started":1}
W20260811 05:13:40.038552 13810 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:40.033505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:43773 (local address 127.28.143.190:39667)
0811 05:13:40.033562 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:40.033564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:40.033674 (+ 110us) server_negotiation.cc:407] Connection header received
0811 05:13:40.033712 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:40.033712 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:40.033724 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:40.033766 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:40.034116 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:40.034620 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:40.035358 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:40.035591 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:40.035657 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:40.038474 (+ 2817us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.143.130:43773: BlockingRecv error: recv got EOF from 127.28.143.130:43773 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 05:13:40.057721 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 12961
I20260811 05:13:40.087950 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 13222
I20260811 05:13:40.107672 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 13461
I20260811 05:13:40.124413 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 12831
2026-08-11T05:13:40Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18035 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T05:13:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:13:40Z Disabled control of system clock
I20260811 05:13:40.159991 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:45261
--webserver_interface=127.28.143.190
--webserver_port=0
--builtin_ntp_servers=127.28.143.148:44145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:45261
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 05:13:40.250240 13817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:40.250479 13817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:40.250500 13817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:40.251791 13817 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:40.251845 13817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:40.251857 13817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:40.251869 13817 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:40.251879 13817 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:13:40.253412 13817 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:44145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:45261
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:45261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13817
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:40.253705 13817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:40.254065 13817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 05:13:40.259626 13817 server_base.cc:1034] running on GCE node
W20260811 05:13:40.259655 13824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:40.259675 13826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:40.259647 13823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:40.260191 13817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:40.260564 13817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:40.261734 13817 hybrid_clock.cc:648] HybridClock initialized: now 1786425220261714 us; error 38 us; skew 500 ppm
I20260811 05:13:40.263067 13817 webserver.cc:460] Webserver started at http://127.28.143.190:37687/ using document root <none> and password file <none>
I20260811 05:13:40.263324 13817 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:40.263376 13817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:40.263546 13817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:40.264735 13817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "7f55a84be7c94be880c311673efd4616"
format_stamp: "Formatted at 2026-08-11 05:13:40 on dist-test-slave-f7th"
server_key: "a6a90cd41457bc3dc5cde033d0bdc96f"
server_key_iv: "d29d303e2c4f92ea48e83699371cded6"
server_key_version: "encryptionkey@0"
I20260811 05:13:40.265098 13817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "7f55a84be7c94be880c311673efd4616"
format_stamp: "Formatted at 2026-08-11 05:13:40 on dist-test-slave-f7th"
server_key: "a6a90cd41457bc3dc5cde033d0bdc96f"
server_key_iv: "d29d303e2c4f92ea48e83699371cded6"
server_key_version: "encryptionkey@0"
I20260811 05:13:40.266516 13817 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 05:13:40.267397 13840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:40.267585 13817 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:40.267683 13817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "7f55a84be7c94be880c311673efd4616"
format_stamp: "Formatted at 2026-08-11 05:13:40 on dist-test-slave-f7th"
server_key: "a6a90cd41457bc3dc5cde033d0bdc96f"
server_key_iv: "d29d303e2c4f92ea48e83699371cded6"
server_key_version: "encryptionkey@0"
I20260811 05:13:40.267747 13817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:40.276494 13817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:40.276861 13817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:40.277004 13817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:40.281436 13817 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:45261
I20260811 05:13:40.281474 13934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:45261 every 8 connection(s)
I20260811 05:13:40.281831 13817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:13:40.282543 13935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:40.285354 13935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f55a84be7c94be880c311673efd4616: Bootstrap starting.
I20260811 05:13:40.285743 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 13817
I20260811 05:13:40.285842 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:13:40.286013 13935 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f55a84be7c94be880c311673efd4616: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:40.286029 29246 external_mini_cluster.cc:1428] Setting key 8c8326fe3e7d9617efe7ca19fa97e345
I20260811 05:13:40.286314 13935 log.cc:824] T 00000000000000000000000000000000 P 7f55a84be7c94be880c311673efd4616: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:40.287048 13935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f55a84be7c94be880c311673efd4616: No bootstrap required, opened a new log
I20260811 05:13:40.288460 13935 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f55a84be7c94be880c311673efd4616 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f55a84be7c94be880c311673efd4616" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 45261 } }
I20260811 05:13:40.288604 13935 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f55a84be7c94be880c311673efd4616 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:40.288663 13935 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f55a84be7c94be880c311673efd4616 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f55a84be7c94be880c311673efd4616, State: Initialized, Role: FOLLOWER
I20260811 05:13:40.288793 13935 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f55a84be7c94be880c311673efd4616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f55a84be7c94be880c311673efd4616" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 45261 } }
I20260811 05:13:40.288888 13935 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f55a84be7c94be880c311673efd4616 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:13:40.288923 13935 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f55a84be7c94be880c311673efd4616 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:13:40.288975 13935 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f55a84be7c94be880c311673efd4616 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:40.289556 13935 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f55a84be7c94be880c311673efd4616 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f55a84be7c94be880c311673efd4616" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 45261 } }
I20260811 05:13:40.289654 13935 leader_election.cc:302] T 00000000000000000000000000000000 P 7f55a84be7c94be880c311673efd4616 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f55a84be7c94be880c311673efd4616; no voters:
I20260811 05:13:40.289819 13935 leader_election.cc:288] T 00000000000000000000000000000000 P 7f55a84be7c94be880c311673efd4616 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:13:40.289860 13949 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f55a84be7c94be880c311673efd4616 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:40.290068 13949 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f55a84be7c94be880c311673efd4616 [term 1 LEADER]: Becoming Leader. State: Replica: 7f55a84be7c94be880c311673efd4616, State: Running, Role: LEADER
I20260811 05:13:40.290104 13935 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f55a84be7c94be880c311673efd4616 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:13:40.290748 13949 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f55a84be7c94be880c311673efd4616 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f55a84be7c94be880c311673efd4616" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 45261 } }
I20260811 05:13:40.291520 13952 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f55a84be7c94be880c311673efd4616 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f55a84be7c94be880c311673efd4616. Latest consensus state: current_term: 1 leader_uuid: "7f55a84be7c94be880c311673efd4616" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f55a84be7c94be880c311673efd4616" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 45261 } } }
I20260811 05:13:40.291757 13952 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f55a84be7c94be880c311673efd4616 [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:40.291805 13955 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f55a84be7c94be880c311673efd4616 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f55a84be7c94be880c311673efd4616" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f55a84be7c94be880c311673efd4616" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 45261 } } }
I20260811 05:13:40.291863 13955 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f55a84be7c94be880c311673efd4616 [sys.catalog]: This master's current role is: LEADER
W20260811 05:13:40.292009 13986 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7f55a84be7c94be880c311673efd4616: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:13:40.292073 13986 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:13:40.292336 13990 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:13:40.292537 13990 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:13:40.292591 13940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:40.286568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46020 (local address 127.28.143.190:45261)
0811 05:13:40.286770 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:40.286778 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:40.286803 (+ 25us) server_negotiation.cc:407] Connection header received
0811 05:13:40.287293 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:40.287300 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:40.287365 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:40.287502 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:40.287725 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:40.289402 (+ 1677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:40.290426 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:40.290975 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:40.291126 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:40.291864 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:40.291868 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:40.291880 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:13:40.292083 (+ 203us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:40.292129 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:13:40.292145 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:40.292206 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:40.292298 (+ 92us) server_negotiation.cc:299] Negotiation successful
0811 05:13:40.292381 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":71,"threads_started":1}
I20260811 05:13:40.296023 13990 catalog_manager.cc:1347] Generated new cluster ID: 011490b9675d4caca52c339e4ca5eee3
I20260811 05:13:40.296093 13990 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:13:40.311482 13990 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:13:40.312039 13990 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:13:40.322311 13990 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f55a84be7c94be880c311673efd4616: Generated new TSK 0
I20260811 05:13:40.322570 13990 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:13:40.327219 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.129:0
--local_ip_for_outbound_sockets=127.28.143.129
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:45261
--builtin_ntp_servers=127.28.143.148:44145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 05:13:40.420861 13994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:40.421129 13994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:40.421154 13994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:40.422595 13994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:40.422670 13994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:40.422700 13994 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.129
I20260811 05:13:40.424454 13994 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:44145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:45261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.129
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:40.424789 13994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:40.425210 13994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:40.431596 14006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:40.431706 14001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:40.431706 14003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:40.432066 13994 server_base.cc:1034] running on GCE node
I20260811 05:13:40.432358 13994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:40.432682 13994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:40.433898 13994 hybrid_clock.cc:648] HybridClock initialized: now 1786425220433880 us; error 33 us; skew 500 ppm
I20260811 05:13:40.435595 13994 webserver.cc:460] Webserver started at http://127.28.143.129:37943/ using document root <none> and password file <none>
I20260811 05:13:40.435856 13994 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:40.435911 13994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:40.436019 13994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:40.437346 13994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/data/instance:
uuid: "074c75586f914bbaaa9dbb7922f832a8"
format_stamp: "Formatted at 2026-08-11 05:13:40 on dist-test-slave-f7th"
server_key: "48751d00c499cec19fc2c80dee52c756"
server_key_iv: "06dbe2a58fe2929ee266b5bf9dd20203"
server_key_version: "encryptionkey@0"
I20260811 05:13:40.437721 13994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance:
uuid: "074c75586f914bbaaa9dbb7922f832a8"
format_stamp: "Formatted at 2026-08-11 05:13:40 on dist-test-slave-f7th"
server_key: "48751d00c499cec19fc2c80dee52c756"
server_key_iv: "06dbe2a58fe2929ee266b5bf9dd20203"
server_key_version: "encryptionkey@0"
I20260811 05:13:40.440088 13994 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:13:40.441422 14014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:40.441990 13994 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 05:13:40.442169 13994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/wal
uuid: "074c75586f914bbaaa9dbb7922f832a8"
format_stamp: "Formatted at 2026-08-11 05:13:40 on dist-test-slave-f7th"
server_key: "48751d00c499cec19fc2c80dee52c756"
server_key_iv: "06dbe2a58fe2929ee266b5bf9dd20203"
server_key_version: "encryptionkey@0"
I20260811 05:13:40.442312 13994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:40.450974 13994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:40.451328 13994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:40.451476 13994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:40.451717 13994 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:40.452080 13994 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:40.452117 13994 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:40.452150 13994 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:40.452167 13994 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:40.459116 13994 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.129:34115
I20260811 05:13:40.459164 14215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.129:34115 every 8 connection(s)
I20260811 05:13:40.459537 13994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
I20260811 05:13:40.464107 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 13994
I20260811 05:13:40.464203 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance
I20260811 05:13:40.464425 29246 external_mini_cluster.cc:1428] Setting key 625f372aeeb3e4ebb5e8e227c478ed7c
I20260811 05:13:40.465348 13940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:40.460517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:59405 (local address 127.28.143.190:45261)
0811 05:13:40.460587 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:40.460589 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:40.460791 (+ 202us) server_negotiation.cc:407] Connection header received
0811 05:13:40.461334 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:40.461336 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:40.461349 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:40.461403 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:40.462154 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:40.462665 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:40.463764 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:40.463994 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:40.464054 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:40.465069 (+ 1015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:40.465071 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:40.465071 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:13:40.465077 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:40.465094 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:13:40.465098 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:40.465143 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:40.465233 (+ 90us) server_negotiation.cc:299] Negotiation successful
0811 05:13:40.465263 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 05:13:40.465579 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.130:0
--local_ip_for_outbound_sockets=127.28.143.130
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:45261
--builtin_ntp_servers=127.28.143.148:44145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 05:13:40.465605 14221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:40.460359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:45261 (local address 127.28.143.129:59405)
0811 05:13:40.460652 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:40.460701 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:40.461224 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:40.461500 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:40.461505 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:40.461515 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:40.462048 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:40.462052 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:40.462769 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:40.462770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:40.463672 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:40.463675 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:40.464084 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:40.464086 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:40.464247 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:40.464909 (+ 662us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:13:40.464909 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:40.464943 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:13:40.464973 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:40.465153 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:40.465159 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:40.465163 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:13:40.465229 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 05:13:40.465352 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":90,"threads_started":1}
I20260811 05:13:40.466240 14217 heartbeater.cc:347] Connected to a master server at 127.28.143.190:45261
I20260811 05:13:40.466372 14217 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:40.466583 14217 heartbeater.cc:511] Master 127.28.143.190:45261 requested a full tablet report, sending...
I20260811 05:13:40.467222 13865 ts_manager.cc:194] Registered new tserver with Master: 074c75586f914bbaaa9dbb7922f832a8 (127.28.143.129:34115)
I20260811 05:13:40.467913 13865 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.143.129:59405
W20260811 05:13:40.554486 14222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:40.554724 14222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:40.554747 14222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:40.555984 14222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:40.556038 14222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:40.556066 14222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.130
I20260811 05:13:40.557520 14222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:44145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:45261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.130
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:40.557780 14222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:40.558125 14222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:40.563776 14238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:40.563782 14240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:40.563781 14237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:40.563820 14222 server_base.cc:1034] running on GCE node
I20260811 05:13:40.564175 14222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:40.564447 14222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:40.565593 14222 hybrid_clock.cc:648] HybridClock initialized: now 1786425220565588 us; error 41 us; skew 500 ppm
I20260811 05:13:40.567130 14222 webserver.cc:460] Webserver started at http://127.28.143.130:35545/ using document root <none> and password file <none>
I20260811 05:13:40.567430 14222 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:40.567487 14222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:40.567624 14222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:40.568832 14222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/data/instance:
uuid: "df336e78d5df49e48d31d52b8fb3c0ea"
format_stamp: "Formatted at 2026-08-11 05:13:40 on dist-test-slave-f7th"
server_key: "758aebfde8a7938cdca0a8a000791fd0"
server_key_iv: "60ee89ae6a984e2d1b54736027bdde0f"
server_key_version: "encryptionkey@0"
I20260811 05:13:40.569198 14222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance:
uuid: "df336e78d5df49e48d31d52b8fb3c0ea"
format_stamp: "Formatted at 2026-08-11 05:13:40 on dist-test-slave-f7th"
server_key: "758aebfde8a7938cdca0a8a000791fd0"
server_key_iv: "60ee89ae6a984e2d1b54736027bdde0f"
server_key_version: "encryptionkey@0"
I20260811 05:13:40.570631 14222 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 05:13:40.571430 14256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:40.571605 14222 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:40.571686 14222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/wal
uuid: "df336e78d5df49e48d31d52b8fb3c0ea"
format_stamp: "Formatted at 2026-08-11 05:13:40 on dist-test-slave-f7th"
server_key: "758aebfde8a7938cdca0a8a000791fd0"
server_key_iv: "60ee89ae6a984e2d1b54736027bdde0f"
server_key_version: "encryptionkey@0"
I20260811 05:13:40.571758 14222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:40.594693 14222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:40.595083 14222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:40.595358 14222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:40.595685 14222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:40.596279 14222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:40.596680 14222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 05:13:40.596745 14222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:40.596773 14222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:40.605540 14222 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.130:36987
I20260811 05:13:40.605635 14467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.130:36987 every 8 connection(s)
I20260811 05:13:40.605975 14222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
I20260811 05:13:40.612052 13940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:40.607061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:54741 (local address 127.28.143.190:45261)
0811 05:13:40.607124 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:40.607126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:40.607360 (+ 234us) server_negotiation.cc:407] Connection header received
0811 05:13:40.607898 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:40.607900 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:40.607911 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:40.607971 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:40.608773 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:40.609304 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:40.610479 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:40.610692 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:40.610758 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:40.611719 (+ 961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:40.611720 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:40.611721 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:13:40.611726 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:40.611742 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:13:40.611746 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:40.611804 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:40.611921 (+ 117us) server_negotiation.cc:299] Negotiation successful
0811 05:13:40.611949 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 05:13:40.612296 14475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:40.607018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:45261 (local address 127.28.143.130:54741)
0811 05:13:40.607231 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:40.607267 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:40.607767 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:40.608071 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:40.608074 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:40.608084 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:40.608604 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:40.608609 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:40.609428 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:40.609429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:40.610316 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:40.610319 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:40.610806 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:40.610807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:40.610941 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:40.611542 (+ 601us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:13:40.611543 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:40.611578 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:13:40.611604 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:40.611838 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:40.611845 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:40.611848 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 05:13:40.611918 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 05:13:40.612057 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":104,"threads_started":1}
I20260811 05:13:40.612882 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 14222
I20260811 05:13:40.612885 14469 heartbeater.cc:347] Connected to a master server at 127.28.143.190:45261
I20260811 05:13:40.612965 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance
I20260811 05:13:40.612977 14469 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:40.613163 29246 external_mini_cluster.cc:1428] Setting key 5fa0c1d7c28db9a6f68a828a2a5335fa
I20260811 05:13:40.613189 14469 heartbeater.cc:511] Master 127.28.143.190:45261 requested a full tablet report, sending...
I20260811 05:13:40.613579 13864 ts_manager.cc:194] Registered new tserver with Master: df336e78d5df49e48d31d52b8fb3c0ea (127.28.143.130:36987)
I20260811 05:13:40.614022 13864 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.143.130:54741
I20260811 05:13:40.614130 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.131:0
--local_ip_for_outbound_sockets=127.28.143.131
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:45261
--builtin_ntp_servers=127.28.143.148:44145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 05:13:40.706599 14478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:40.706856 14478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:40.706880 14478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:40.708102 14478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:40.708158 14478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:40.708184 14478 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.131
I20260811 05:13:40.709632 14478 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:44145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:45261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14478
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.131
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:40.709877 14478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:40.710243 14478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 05:13:40.716182 14478 server_base.cc:1034] running on GCE node
W20260811 05:13:40.716157 14485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:40.716132 14486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:40.716144 14488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:40.716570 14478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:40.716854 14478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:40.718056 14478 hybrid_clock.cc:648] HybridClock initialized: now 1786425220718020 us; error 55 us; skew 500 ppm
I20260811 05:13:40.719538 14478 webserver.cc:460] Webserver started at http://127.28.143.131:36731/ using document root <none> and password file <none>
I20260811 05:13:40.719812 14478 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:40.719877 14478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:40.720017 14478 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:40.721184 14478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/data/instance:
uuid: "533669d930134f48b8d43b402ea866c6"
format_stamp: "Formatted at 2026-08-11 05:13:40 on dist-test-slave-f7th"
server_key: "3543a345133729c90b27c78b237cf71a"
server_key_iv: "4086dfe5a198d4b92a78c7df74167a4f"
server_key_version: "encryptionkey@0"
I20260811 05:13:40.721539 14478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance:
uuid: "533669d930134f48b8d43b402ea866c6"
format_stamp: "Formatted at 2026-08-11 05:13:40 on dist-test-slave-f7th"
server_key: "3543a345133729c90b27c78b237cf71a"
server_key_iv: "4086dfe5a198d4b92a78c7df74167a4f"
server_key_version: "encryptionkey@0"
I20260811 05:13:40.722911 14478 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 05:13:40.723752 14495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:40.724020 14478 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:40.724118 14478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/wal
uuid: "533669d930134f48b8d43b402ea866c6"
format_stamp: "Formatted at 2026-08-11 05:13:40 on dist-test-slave-f7th"
server_key: "3543a345133729c90b27c78b237cf71a"
server_key_iv: "4086dfe5a198d4b92a78c7df74167a4f"
server_key_version: "encryptionkey@0"
I20260811 05:13:40.724188 14478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:40.741250 14478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:40.741601 14478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:40.741750 14478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:40.741999 14478 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:40.742352 14478 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:40.742386 14478 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:40.742419 14478 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:40.742436 14478 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:40.748396 14478 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.131:34827
I20260811 05:13:40.748440 14693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.131:34827 every 8 connection(s)
I20260811 05:13:40.748818 14478 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
I20260811 05:13:40.750846 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 14478
I20260811 05:13:40.750936 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance
I20260811 05:13:40.751116 29246 external_mini_cluster.cc:1428] Setting key 1f69896f391d03e3210deda10956dd30
I20260811 05:13:40.754458 13940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:40.749719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:50177 (local address 127.28.143.190:45261)
0811 05:13:40.749813 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:40.749816 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:40.749980 (+ 164us) server_negotiation.cc:407] Connection header received
0811 05:13:40.750537 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:40.750539 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:40.750552 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:40.750605 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:40.751381 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:40.752087 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:40.753151 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:40.753335 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:40.753384 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:40.754146 (+ 762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:40.754147 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:40.754147 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:13:40.754152 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:40.754172 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:13:40.754176 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:40.754217 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:40.754324 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 05:13:40.754350 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 05:13:40.754527 14699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:40.749606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:45261 (local address 127.28.143.131:50177)
0811 05:13:40.749856 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:40.749894 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:40.750432 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:40.750694 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:40.750697 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:40.750706 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:40.751272 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:40.751280 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:40.752179 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:40.752180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:40.753072 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:40.753075 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:40.753403 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:40.753404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:40.753518 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:40.754018 (+ 500us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:13:40.754018 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:40.754049 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:13:40.754072 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:40.754225 (+ 153us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:40.754227 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:40.754230 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 05:13:40.754279 (+ 49us) client_negotiation.cc:240] Negotiation successful
0811 05:13:40.754360 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":86,"threads_started":1}
I20260811 05:13:40.755003 14694 heartbeater.cc:347] Connected to a master server at 127.28.143.190:45261
I20260811 05:13:40.755113 14694 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:40.755291 14694 heartbeater.cc:511] Master 127.28.143.190:45261 requested a full tablet report, sending...
I20260811 05:13:40.755708 13864 ts_manager.cc:194] Registered new tserver with Master: 533669d930134f48b8d43b402ea866c6 (127.28.143.131:34827)
I20260811 05:13:40.756076 13864 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.143.131:50177
I20260811 05:13:40.763761 29246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 05:13:40.765882 13940 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:40.765276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60988 (local address 127.28.143.190:45261)
0811 05:13:40.765338 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:40.765341 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:40.765466 (+ 125us) server_negotiation.cc:407] Connection header received
0811 05:13:40.765580 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:40.765582 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:40.765593 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:40.765639 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:40.765802 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60988: BlockingRecv error: recv got EOF from 127.0.0.1:60988 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 05:13:40.766031 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 13994
I20260811 05:13:40.814591 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 14222
I20260811 05:13:40.881060 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 14478
I20260811 05:13:40.932507 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 13817
2026-08-11T05:13:40Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (811 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T05:13:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:13:40Z Disabled control of system clock
I20260811 05:13:40.973382 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:39993
--webserver_interface=127.28.143.190
--webserver_port=0
--builtin_ntp_servers=127.28.143.148:41993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:39993
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 05:13:41.070128 14716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:41.070371 14716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:41.070395 14716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:41.071980 14716 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:41.072054 14716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:41.072067 14716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:41.072079 14716 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:41.072095 14716 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:13:41.073879 14716 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:41993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:39993
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:39993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:41.074182 14716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:41.074513 14716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:41.079546 14725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:41.079560 14716 server_base.cc:1034] running on GCE node
W20260811 05:13:41.079530 14723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:41.079528 14732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:41.080006 14716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:41.080299 14716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:41.081470 14716 hybrid_clock.cc:648] HybridClock initialized: now 1786425221081457 us; error 33 us; skew 500 ppm
I20260811 05:13:41.082777 14716 webserver.cc:460] Webserver started at http://127.28.143.190:40561/ using document root <none> and password file <none>
I20260811 05:13:41.083050 14716 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:41.083107 14716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:41.083238 14716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:41.084324 14716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "be114e566051441f8e3a28012cde8412"
format_stamp: "Formatted at 2026-08-11 05:13:41 on dist-test-slave-f7th"
server_key: "cd8ecc21f7743e53136ce4529e21f7af"
server_key_iv: "3c4be1490dbe7d920a8a539438d97ef7"
server_key_version: "encryptionkey@0"
I20260811 05:13:41.084702 14716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "be114e566051441f8e3a28012cde8412"
format_stamp: "Formatted at 2026-08-11 05:13:41 on dist-test-slave-f7th"
server_key: "cd8ecc21f7743e53136ce4529e21f7af"
server_key_iv: "3c4be1490dbe7d920a8a539438d97ef7"
server_key_version: "encryptionkey@0"
I20260811 05:13:41.085992 14716 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:41.086859 14744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:41.087095 14716 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:41.087200 14716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "be114e566051441f8e3a28012cde8412"
format_stamp: "Formatted at 2026-08-11 05:13:41 on dist-test-slave-f7th"
server_key: "cd8ecc21f7743e53136ce4529e21f7af"
server_key_iv: "3c4be1490dbe7d920a8a539438d97ef7"
server_key_version: "encryptionkey@0"
I20260811 05:13:41.087268 14716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:41.091080 14716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:41.091388 14716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:41.091522 14716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:41.095723 14716 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:39993
I20260811 05:13:41.095746 14835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:39993 every 8 connection(s)
I20260811 05:13:41.096267 14716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:13:41.097112 14836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:41.099849 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 14716
I20260811 05:13:41.099954 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:13:41.100133 29246 external_mini_cluster.cc:1428] Setting key e7a4e60bdd5e14793946ce78b40bdd85
I20260811 05:13:41.100145 14836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be114e566051441f8e3a28012cde8412: Bootstrap starting.
I20260811 05:13:41.100862 14836 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be114e566051441f8e3a28012cde8412: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:41.101174 14836 log.cc:824] T 00000000000000000000000000000000 P be114e566051441f8e3a28012cde8412: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:41.101854 14836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be114e566051441f8e3a28012cde8412: No bootstrap required, opened a new log
I20260811 05:13:41.102666 14839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:41.100713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57908 (local address 127.28.143.190:39993)
0811 05:13:41.100913 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:41.100920 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:41.100945 (+ 25us) server_negotiation.cc:407] Connection header received
0811 05:13:41.101431 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:41.101438 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:41.101477 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:41.101618 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:41.102274 (+ 656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:41.102277 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:41.102278 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:13:41.102282 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:41.102322 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:13:41.102332 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:41.102408 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:41.102448 (+ 40us) server_negotiation.cc:299] Negotiation successful
0811 05:13:41.102516 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":82,"threads_started":1}
I20260811 05:13:41.103484 14836 raft_consensus.cc:348] T 00000000000000000000000000000000 P be114e566051441f8e3a28012cde8412 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be114e566051441f8e3a28012cde8412" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39993 } }
I20260811 05:13:41.103618 14836 raft_consensus.cc:374] T 00000000000000000000000000000000 P be114e566051441f8e3a28012cde8412 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:41.103639 14836 raft_consensus.cc:729] T 00000000000000000000000000000000 P be114e566051441f8e3a28012cde8412 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be114e566051441f8e3a28012cde8412, State: Initialized, Role: FOLLOWER
I20260811 05:13:41.103751 14836 consensus_queue.cc:260] T 00000000000000000000000000000000 P be114e566051441f8e3a28012cde8412 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be114e566051441f8e3a28012cde8412" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39993 } }
I20260811 05:13:41.103816 14836 raft_consensus.cc:388] T 00000000000000000000000000000000 P be114e566051441f8e3a28012cde8412 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:13:41.103837 14836 raft_consensus.cc:482] T 00000000000000000000000000000000 P be114e566051441f8e3a28012cde8412 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:13:41.103891 14836 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be114e566051441f8e3a28012cde8412 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:41.104735 14836 raft_consensus.cc:504] T 00000000000000000000000000000000 P be114e566051441f8e3a28012cde8412 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be114e566051441f8e3a28012cde8412" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39993 } }
I20260811 05:13:41.104827 14836 leader_election.cc:302] T 00000000000000000000000000000000 P be114e566051441f8e3a28012cde8412 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be114e566051441f8e3a28012cde8412; no voters:
I20260811 05:13:41.104980 14836 leader_election.cc:288] T 00000000000000000000000000000000 P be114e566051441f8e3a28012cde8412 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:13:41.105026 14841 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be114e566051441f8e3a28012cde8412 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:41.105228 14841 raft_consensus.cc:686] T 00000000000000000000000000000000 P be114e566051441f8e3a28012cde8412 [term 1 LEADER]: Becoming Leader. State: Replica: be114e566051441f8e3a28012cde8412, State: Running, Role: LEADER
I20260811 05:13:41.105332 14836 sys_catalog.cc:564] T 00000000000000000000000000000000 P be114e566051441f8e3a28012cde8412 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:13:41.105382 14841 consensus_queue.cc:237] T 00000000000000000000000000000000 P be114e566051441f8e3a28012cde8412 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be114e566051441f8e3a28012cde8412" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39993 } }
I20260811 05:13:41.105803 14844 sys_catalog.cc:455] T 00000000000000000000000000000000 P be114e566051441f8e3a28012cde8412 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "be114e566051441f8e3a28012cde8412" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be114e566051441f8e3a28012cde8412" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39993 } } }
I20260811 05:13:41.105911 14844 sys_catalog.cc:458] T 00000000000000000000000000000000 P be114e566051441f8e3a28012cde8412 [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:41.106081 14845 sys_catalog.cc:455] T 00000000000000000000000000000000 P be114e566051441f8e3a28012cde8412 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be114e566051441f8e3a28012cde8412. Latest consensus state: current_term: 1 leader_uuid: "be114e566051441f8e3a28012cde8412" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be114e566051441f8e3a28012cde8412" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39993 } } }
I20260811 05:13:41.106113 14854 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:13:41.106135 14845 sys_catalog.cc:458] T 00000000000000000000000000000000 P be114e566051441f8e3a28012cde8412 [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:41.106623 14854 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:13:41.108225 14854 catalog_manager.cc:1347] Generated new cluster ID: 6071cd64d65c476e9e9071d63a47f005
I20260811 05:13:41.108290 14854 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:13:41.120131 14854 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:13:41.120328 14854 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:13:41.128194 14854 catalog_manager.cc:5868] T 00000000000000000000000000000000 P be114e566051441f8e3a28012cde8412: Generated new TSK 0
I20260811 05:13:41.128450 14854 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:13:41.131206 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.129:0
--local_ip_for_outbound_sockets=127.28.143.129
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:39993
--builtin_ntp_servers=127.28.143.148:41993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 05:13:41.222725 14875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:41.222992 14875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:41.223019 14875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:41.224426 14875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:41.224485 14875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:41.224516 14875 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.129
I20260811 05:13:41.226099 14875 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:41993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:39993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.129
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:41.226408 14875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:41.226734 14875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:41.233596 14891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:41.233603 14900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:41.233986 14897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:41.234526 14875 server_base.cc:1034] running on GCE node
I20260811 05:13:41.234787 14875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:41.235221 14875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:41.236393 14875 hybrid_clock.cc:648] HybridClock initialized: now 1786425221236365 us; error 46 us; skew 500 ppm
I20260811 05:13:41.238121 14875 webserver.cc:460] Webserver started at http://127.28.143.129:34995/ using document root <none> and password file <none>
I20260811 05:13:41.238425 14875 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:41.238487 14875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:41.238632 14875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:41.240331 14875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/data/instance:
uuid: "a4d12444784343f099f150725a7192c5"
format_stamp: "Formatted at 2026-08-11 05:13:41 on dist-test-slave-f7th"
server_key: "1f9f5c747caf8032c4a803130f7df129"
server_key_iv: "808e63e871e61cd43c08e1fd2a793167"
server_key_version: "encryptionkey@0"
I20260811 05:13:41.240856 14875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance:
uuid: "a4d12444784343f099f150725a7192c5"
format_stamp: "Formatted at 2026-08-11 05:13:41 on dist-test-slave-f7th"
server_key: "1f9f5c747caf8032c4a803130f7df129"
server_key_iv: "808e63e871e61cd43c08e1fd2a793167"
server_key_version: "encryptionkey@0"
I20260811 05:13:41.242659 14875 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260811 05:13:41.243765 14912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:41.244341 14875 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:41.244664 14875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/wal
uuid: "a4d12444784343f099f150725a7192c5"
format_stamp: "Formatted at 2026-08-11 05:13:41 on dist-test-slave-f7th"
server_key: "1f9f5c747caf8032c4a803130f7df129"
server_key_iv: "808e63e871e61cd43c08e1fd2a793167"
server_key_version: "encryptionkey@0"
I20260811 05:13:41.244809 14875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:41.248977 14875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:41.249311 14875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:41.249461 14875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:41.249699 14875 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:41.250053 14875 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:41.250090 14875 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:41.250121 14875 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:41.250138 14875 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:41.257970 14875 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.129:39073
I20260811 05:13:41.258045 15125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.129:39073 every 8 connection(s)
I20260811 05:13:41.258397 14875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
I20260811 05:13:41.262936 14839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:41.259457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:58051 (local address 127.28.143.190:39993)
0811 05:13:41.259544 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:41.259547 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:41.259967 (+ 420us) server_negotiation.cc:407] Connection header received
0811 05:13:41.260783 (+ 816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:41.260786 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:41.260795 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:41.260882 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:41.262635 (+ 1753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:41.262637 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:41.262637 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:13:41.262643 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:41.262662 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:13:41.262667 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:41.262733 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:41.262858 (+ 125us) server_negotiation.cc:299] Negotiation successful
0811 05:13:41.262872 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 05:13:41.263171 15130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:41.259252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39993 (local address 127.28.143.129:58051)
0811 05:13:41.259675 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:41.259736 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:41.260408 (+ 672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:41.261102 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:41.261109 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:41.261121 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:41.262371 (+ 1250us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:13:41.262372 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:41.262414 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:13:41.262455 (+ 41us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:41.262767 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:41.262776 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:41.262779 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 05:13:41.262868 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 05:13:41.262949 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":80,"threads_started":1}
I20260811 05:13:41.263790 15126 heartbeater.cc:347] Connected to a master server at 127.28.143.190:39993
I20260811 05:13:41.263898 15126 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:41.264081 15126 heartbeater.cc:511] Master 127.28.143.190:39993 requested a full tablet report, sending...
I20260811 05:13:41.264724 14776 ts_manager.cc:194] Registered new tserver with Master: a4d12444784343f099f150725a7192c5 (127.28.143.129:39073)
I20260811 05:13:41.268193 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 14875
I20260811 05:13:41.268333 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance
I20260811 05:13:41.268566 29246 external_mini_cluster.cc:1428] Setting key 35b5765e5685aa18ee8229392557db03
I20260811 05:13:41.270112 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.130:0
--local_ip_for_outbound_sockets=127.28.143.130
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:39993
--builtin_ntp_servers=127.28.143.148:41993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 05:13:41.359751 15132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:41.360005 15132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:41.360042 15132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:41.361563 15132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:41.361630 15132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:41.361670 15132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.130
I20260811 05:13:41.363308 15132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:41993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:39993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.130
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:41.363765 15132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:41.364166 15132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:41.370503 15150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:41.370445 15145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:41.370641 15132 server_base.cc:1034] running on GCE node
W20260811 05:13:41.370445 15143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:41.371249 15132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:41.371542 15132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:41.372706 15132 hybrid_clock.cc:648] HybridClock initialized: now 1786425221372671 us; error 54 us; skew 500 ppm
I20260811 05:13:41.374203 15132 webserver.cc:460] Webserver started at http://127.28.143.130:39239/ using document root <none> and password file <none>
I20260811 05:13:41.374432 15132 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:41.374482 15132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:41.374625 15132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:41.375746 15132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/data/instance:
uuid: "e01fbc084a074774a878215dea9faaec"
format_stamp: "Formatted at 2026-08-11 05:13:41 on dist-test-slave-f7th"
server_key: "37abba755dcefa613654d2f99f67a977"
server_key_iv: "c712a22d37fd7fb6c5adc8161ca58cd2"
server_key_version: "encryptionkey@0"
I20260811 05:13:41.376083 15132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance:
uuid: "e01fbc084a074774a878215dea9faaec"
format_stamp: "Formatted at 2026-08-11 05:13:41 on dist-test-slave-f7th"
server_key: "37abba755dcefa613654d2f99f67a977"
server_key_iv: "c712a22d37fd7fb6c5adc8161ca58cd2"
server_key_version: "encryptionkey@0"
I20260811 05:13:41.377508 15132 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 05:13:41.378283 15165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:41.378438 15132 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 05:13:41.378528 15132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/wal
uuid: "e01fbc084a074774a878215dea9faaec"
format_stamp: "Formatted at 2026-08-11 05:13:41 on dist-test-slave-f7th"
server_key: "37abba755dcefa613654d2f99f67a977"
server_key_iv: "c712a22d37fd7fb6c5adc8161ca58cd2"
server_key_version: "encryptionkey@0"
I20260811 05:13:41.378597 15132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:41.382113 15132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:41.382387 15132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:41.382526 15132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:41.382732 15132 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:41.383050 15132 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:41.383083 15132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:41.383113 15132 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:41.383138 15132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:41.389183 15132 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.130:42749
I20260811 05:13:41.389249 15374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.130:42749 every 8 connection(s)
I20260811 05:13:41.389664 15132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
I20260811 05:13:41.393095 14839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:41.390473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:53361 (local address 127.28.143.190:39993)
0811 05:13:41.390565 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:41.390568 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:41.390790 (+ 222us) server_negotiation.cc:407] Connection header received
0811 05:13:41.391349 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:41.391351 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:41.391358 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:41.391403 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:41.392689 (+ 1286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:41.392691 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:41.392692 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:13:41.392697 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:41.392716 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:13:41.392720 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:41.392771 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:41.392949 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 05:13:41.393010 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 05:13:41.393252 15377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:41.390424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39993 (local address 127.28.143.130:53361)
0811 05:13:41.390666 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:41.390704 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:41.391236 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:41.391478 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:41.391483 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:41.391494 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:41.392450 (+ 956us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:13:41.392450 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:41.392487 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:13:41.392522 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:41.392777 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:41.392784 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:41.392788 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:13:41.392951 (+ 163us) client_negotiation.cc:240] Negotiation successful
0811 05:13:41.393026 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":91,"threads_started":1}
I20260811 05:13:41.393781 15375 heartbeater.cc:347] Connected to a master server at 127.28.143.190:39993
I20260811 05:13:41.393875 15375 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:41.394029 15375 heartbeater.cc:511] Master 127.28.143.190:39993 requested a full tablet report, sending...
I20260811 05:13:41.394439 14776 ts_manager.cc:194] Registered new tserver with Master: e01fbc084a074774a878215dea9faaec (127.28.143.130:42749)
I20260811 05:13:41.396906 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 15132
I20260811 05:13:41.397002 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance
I20260811 05:13:41.397197 29246 external_mini_cluster.cc:1428] Setting key 1d81905f77e4d04b1c7ef8d3b54d835d
I20260811 05:13:41.398370 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.131:0
--local_ip_for_outbound_sockets=127.28.143.131
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:39993
--builtin_ntp_servers=127.28.143.148:41993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 05:13:41.491510 15379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:41.491758 15379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:41.491783 15379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:41.493126 15379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:41.493187 15379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:41.493216 15379 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.131
I20260811 05:13:41.495077 15379 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:41993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:39993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15379
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.131
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:41.495388 15379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:41.495743 15379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:41.501327 15390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:41.501515 15391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:41.501751 15379 server_base.cc:1034] running on GCE node
W20260811 05:13:41.501904 15393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:41.502161 15379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:41.502450 15379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:41.503665 15379 hybrid_clock.cc:648] HybridClock initialized: now 1786425221503646 us; error 32 us; skew 500 ppm
I20260811 05:13:41.505167 15379 webserver.cc:460] Webserver started at http://127.28.143.131:35197/ using document root <none> and password file <none>
I20260811 05:13:41.505457 15379 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:41.505523 15379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:41.505672 15379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:41.506791 15379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/data/instance:
uuid: "d728c6e5953d4b13bf8af098a6f8d561"
format_stamp: "Formatted at 2026-08-11 05:13:41 on dist-test-slave-f7th"
server_key: "69660bf65a8edfc902bc91e3989ecfda"
server_key_iv: "ababf12a5f6b909ce40a3826bf1c9d3b"
server_key_version: "encryptionkey@0"
I20260811 05:13:41.507134 15379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance:
uuid: "d728c6e5953d4b13bf8af098a6f8d561"
format_stamp: "Formatted at 2026-08-11 05:13:41 on dist-test-slave-f7th"
server_key: "69660bf65a8edfc902bc91e3989ecfda"
server_key_iv: "ababf12a5f6b909ce40a3826bf1c9d3b"
server_key_version: "encryptionkey@0"
I20260811 05:13:41.508503 15379 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 05:13:41.509469 15406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:41.509671 15379 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:41.509757 15379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/wal
uuid: "d728c6e5953d4b13bf8af098a6f8d561"
format_stamp: "Formatted at 2026-08-11 05:13:41 on dist-test-slave-f7th"
server_key: "69660bf65a8edfc902bc91e3989ecfda"
server_key_iv: "ababf12a5f6b909ce40a3826bf1c9d3b"
server_key_version: "encryptionkey@0"
I20260811 05:13:41.509819 15379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:41.513221 15379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:41.513487 15379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:41.513597 15379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:41.513777 15379 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:41.514048 15379 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:41.514079 15379 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:41.514099 15379 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:41.514109 15379 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:41.519726 15379 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.131:39465
I20260811 05:13:41.519802 15604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.131:39465 every 8 connection(s)
I20260811 05:13:41.520128 15379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
I20260811 05:13:41.523471 14839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:41.521037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:41935 (local address 127.28.143.190:39993)
0811 05:13:41.521127 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:41.521130 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:41.521337 (+ 207us) server_negotiation.cc:407] Connection header received
0811 05:13:41.521838 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:41.521839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:41.521847 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:41.521918 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:41.523216 (+ 1298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:41.523218 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:41.523218 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:13:41.523223 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:41.523241 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:13:41.523245 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:41.523299 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:41.523371 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 05:13:41.523408 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 05:13:41.523638 15611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:41.520988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:39993 (local address 127.28.143.131:41935)
0811 05:13:41.521189 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:41.521229 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:41.521710 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:41.522045 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:41.522050 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:41.522060 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:41.523043 (+ 983us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:13:41.523043 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:41.523076 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:13:41.523109 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:41.523311 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:41.523314 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:41.523316 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 05:13:41.523371 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 05:13:41.523428 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":90,"threads_started":1}
I20260811 05:13:41.524296 15606 heartbeater.cc:347] Connected to a master server at 127.28.143.190:39993
I20260811 05:13:41.524398 15606 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:41.524534 15606 heartbeater.cc:511] Master 127.28.143.190:39993 requested a full tablet report, sending...
I20260811 05:13:41.524998 14776 ts_manager.cc:194] Registered new tserver with Master: d728c6e5953d4b13bf8af098a6f8d561 (127.28.143.131:39465)
I20260811 05:13:41.525200 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 15379
I20260811 05:13:41.525312 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance
I20260811 05:13:41.525516 29246 external_mini_cluster.cc:1428] Setting key 434c21dc70a4f5e32896bbc9b2b4e5f0
I20260811 05:13:41.526762 29246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 05:13:41.528790 14839 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:41.528317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57916 (local address 127.28.143.190:39993)
0811 05:13:41.528398 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:41.528400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:41.528445 (+ 45us) server_negotiation.cc:407] Connection header received
0811 05:13:41.528490 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:41.528491 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:41.528496 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:41.528555 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:41.528714 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57916: BlockingRecv error: recv got EOF from 127.0.0.1:57916 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 05:13:41.528908 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 14875
I20260811 05:13:41.551798 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 15132
I20260811 05:13:41.592692 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 15379
I20260811 05:13:41.615938 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 14716
2026-08-11T05:13:41Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (692 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:13:41 dist-test-slave-f7th krb5kdc[15622](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:13:41 dist-test-slave-f7th krb5kdc[15622](info): set up 2 sockets
Aug 11 05:13:41 dist-test-slave-f7th krb5kdc[15622](info): commencing operation
krb5kdc: starting...
W20260811 05:13:42.861099 29246 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Aug 11 05:13:42 dist-test-slave-f7th krb5kdc[15622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425222, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:13:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:13:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
I20260811 05:13:43.006115 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:41661
--webserver_interface=127.28.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:35655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:41661
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:43.095592 15646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:43.095842 15646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:43.095877 15646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:43.097258 15646 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:43.097321 15646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:43.097340 15646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:43.097357 15646 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:43.097373 15646 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:13:43.099144 15646 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:35655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:41661
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:41661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:43.099457 15646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:43.100045 15646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:43.107110 15652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:43.107120 15654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:43.107385 15646 server_base.cc:1034] running on GCE node
W20260811 05:13:43.107120 15656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:43.107852 15646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:43.108242 15646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:43.109431 15646 hybrid_clock.cc:648] HybridClock initialized: now 1786425223109413 us; error 39 us; skew 500 ppm
Aug 11 05:13:43 dist-test-slave-f7th krb5kdc[15622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425223, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:43.112391 15646 init.cc:377] Logged in from keytab as kudu/127.28.143.190@KRBTEST.COM (short username kudu)
I20260811 05:13:43.113085 15646 webserver.cc:460] Webserver started at http://127.28.143.190:42051/ using document root <none> and password file <none>
I20260811 05:13:43.113364 15646 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:43.113425 15646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:43.113607 15646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:43.114831 15646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "8477996b1b214cde87d4faef02ecffaf"
format_stamp: "Formatted at 2026-08-11 05:13:43 on dist-test-slave-f7th"
server_key: "47cc4a578cad21b1a78693a7f062bad9"
server_key_iv: "686c3046c7dad9492dfb7f0b44fb028c"
server_key_version: "encryptionkey@0"
I20260811 05:13:43.115203 15646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "8477996b1b214cde87d4faef02ecffaf"
format_stamp: "Formatted at 2026-08-11 05:13:43 on dist-test-slave-f7th"
server_key: "47cc4a578cad21b1a78693a7f062bad9"
server_key_iv: "686c3046c7dad9492dfb7f0b44fb028c"
server_key_version: "encryptionkey@0"
I20260811 05:13:43.116606 15646 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 05:13:43.117558 15668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:43.117762 15646 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 05:13:43.117868 15646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "8477996b1b214cde87d4faef02ecffaf"
format_stamp: "Formatted at 2026-08-11 05:13:43 on dist-test-slave-f7th"
server_key: "47cc4a578cad21b1a78693a7f062bad9"
server_key_iv: "686c3046c7dad9492dfb7f0b44fb028c"
server_key_version: "encryptionkey@0"
I20260811 05:13:43.117939 15646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:43.130309 15646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:43.132905 15646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:43.133075 15646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:43.137252 15646 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:41661
I20260811 05:13:43.137303 15762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:41661 every 8 connection(s)
I20260811 05:13:43.137666 15646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:13:43.139143 15763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:43.141759 15763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8477996b1b214cde87d4faef02ecffaf: Bootstrap starting.
I20260811 05:13:43.142365 15763 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8477996b1b214cde87d4faef02ecffaf: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:43.142621 15763 log.cc:824] T 00000000000000000000000000000000 P 8477996b1b214cde87d4faef02ecffaf: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:43.143553 15763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8477996b1b214cde87d4faef02ecffaf: No bootstrap required, opened a new log
I20260811 05:13:43.144279 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 15646
I20260811 05:13:43.144366 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:13:43.144536 29246 external_mini_cluster.cc:1428] Setting key 6de6607da6870b9b8dacb98dda4890f3
I20260811 05:13:43.145751 15763 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8477996b1b214cde87d4faef02ecffaf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8477996b1b214cde87d4faef02ecffaf" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 41661 } }
I20260811 05:13:43.145929 15763 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8477996b1b214cde87d4faef02ecffaf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:43.145989 15763 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8477996b1b214cde87d4faef02ecffaf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8477996b1b214cde87d4faef02ecffaf, State: Initialized, Role: FOLLOWER
I20260811 05:13:43.146185 15763 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8477996b1b214cde87d4faef02ecffaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8477996b1b214cde87d4faef02ecffaf" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 41661 } }
I20260811 05:13:43.146287 15763 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8477996b1b214cde87d4faef02ecffaf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:13:43.146322 15763 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8477996b1b214cde87d4faef02ecffaf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:13:43.146376 15763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8477996b1b214cde87d4faef02ecffaf [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:43.147522 15763 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8477996b1b214cde87d4faef02ecffaf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8477996b1b214cde87d4faef02ecffaf" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 41661 } }
I20260811 05:13:43.147642 15763 leader_election.cc:302] T 00000000000000000000000000000000 P 8477996b1b214cde87d4faef02ecffaf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8477996b1b214cde87d4faef02ecffaf; no voters:
I20260811 05:13:43.147819 15763 leader_election.cc:288] T 00000000000000000000000000000000 P 8477996b1b214cde87d4faef02ecffaf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:13:43.147861 15776 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8477996b1b214cde87d4faef02ecffaf [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:43.147977 15776 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8477996b1b214cde87d4faef02ecffaf [term 1 LEADER]: Becoming Leader. State: Replica: 8477996b1b214cde87d4faef02ecffaf, State: Running, Role: LEADER
I20260811 05:13:43.148128 15763 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8477996b1b214cde87d4faef02ecffaf [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:13:43.148132 15776 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8477996b1b214cde87d4faef02ecffaf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8477996b1b214cde87d4faef02ecffaf" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 41661 } }
I20260811 05:13:43.148687 15781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8477996b1b214cde87d4faef02ecffaf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8477996b1b214cde87d4faef02ecffaf. Latest consensus state: current_term: 1 leader_uuid: "8477996b1b214cde87d4faef02ecffaf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8477996b1b214cde87d4faef02ecffaf" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 41661 } } }
I20260811 05:13:43.148779 15781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8477996b1b214cde87d4faef02ecffaf [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:43.149148 15779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8477996b1b214cde87d4faef02ecffaf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8477996b1b214cde87d4faef02ecffaf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8477996b1b214cde87d4faef02ecffaf" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 41661 } } }
I20260811 05:13:43.149214 15779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8477996b1b214cde87d4faef02ecffaf [sys.catalog]: This master's current role is: LEADER
W20260811 05:13:43.150012 15801 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8477996b1b214cde87d4faef02ecffaf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:13:43.150236 15801 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:13:43.150334 15791 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:13:43.150519 15791 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 11 05:13:43 dist-test-slave-f7th krb5kdc[15622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425222, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:13:43.152280 15791 catalog_manager.cc:1347] Generated new cluster ID: a2c9c1df4955402fb3f0b381e00cbfd9
I20260811 05:13:43.152338 15791 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:13:43.156100 15771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:43.145195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35442 (local address 127.28.143.190:41661)
0811 05:13:43.145379 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:43.145389 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:43.145416 (+ 27us) server_negotiation.cc:407] Connection header received
0811 05:13:43.145798 (+ 382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:43.145810 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:43.145890 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:43.146078 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:43.146891 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.147957 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.148951 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.149574 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.149686 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:43.152411 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:43.152416 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:43.152417 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:43.152430 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:43.154049 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:43.154409 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:43.154414 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:43.154414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:43.154527 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:43.154849 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:43.154850 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:43.154850 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:43.155116 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:43.155297 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:43.155740 (+ 443us) server_negotiation.cc:299] Negotiation successful
0811 05:13:43.155917 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":81,"threads_started":1}
I20260811 05:13:43.159446 15791 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:13:43.160044 15791 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:13:43.162827 15791 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8477996b1b214cde87d4faef02ecffaf: Generated new TSK 0
I20260811 05:13:43.163050 15791 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.143.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab.
Entry for principal kudu/127.28.143.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab.
I20260811 05:13:43.204212 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.129:0
--local_ip_for_outbound_sockets=127.28.143.129
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:41661
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:35655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:43.306646 15809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:43.306938 15809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:43.306978 15809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:43.308549 15809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:43.308684 15809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:43.308867 15809 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.129
I20260811 05:13:43.311026 15809 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:35655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:41661
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15809
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.129
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:43.311357 15809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:43.311897 15809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:43.318528 15818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:43.318614 15821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:43.318591 15819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:43.318673 15809 server_base.cc:1034] running on GCE node
I20260811 05:13:43.319010 15809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:43.319314 15809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:43.320497 15809 hybrid_clock.cc:648] HybridClock initialized: now 1786425223320478 us; error 43 us; skew 500 ppm
Aug 11 05:13:43 dist-test-slave-f7th krb5kdc[15622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425223, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:43.323494 15809 init.cc:377] Logged in from keytab as kudu/127.28.143.129@KRBTEST.COM (short username kudu)
I20260811 05:13:43.324007 15809 webserver.cc:460] Webserver started at http://127.28.143.129:39069/ using document root <none> and password file <none>
I20260811 05:13:43.324232 15809 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:43.324285 15809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:43.324429 15809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:43.325623 15809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/data/instance:
uuid: "12f8c822fb7a4fc58bede437ec1166a5"
format_stamp: "Formatted at 2026-08-11 05:13:43 on dist-test-slave-f7th"
server_key: "1940de15ec1907ff11d959830943ccdf"
server_key_iv: "e52654d06b28f241f1af743ca8691bcf"
server_key_version: "encryptionkey@0"
I20260811 05:13:43.326005 15809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance:
uuid: "12f8c822fb7a4fc58bede437ec1166a5"
format_stamp: "Formatted at 2026-08-11 05:13:43 on dist-test-slave-f7th"
server_key: "1940de15ec1907ff11d959830943ccdf"
server_key_iv: "e52654d06b28f241f1af743ca8691bcf"
server_key_version: "encryptionkey@0"
I20260811 05:13:43.327539 15809 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 05:13:43.328493 15838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:43.328722 15809 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:43.328819 15809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/wal
uuid: "12f8c822fb7a4fc58bede437ec1166a5"
format_stamp: "Formatted at 2026-08-11 05:13:43 on dist-test-slave-f7th"
server_key: "1940de15ec1907ff11d959830943ccdf"
server_key_iv: "e52654d06b28f241f1af743ca8691bcf"
server_key_version: "encryptionkey@0"
I20260811 05:13:43.328894 15809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:43.341820 15809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:43.343930 15809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:43.344082 15809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:43.344316 15809 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:43.344686 15809 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:43.344720 15809 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:43.344750 15809 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:43.344769 15809 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:43.350905 15809 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.129:39265
I20260811 05:13:43.350950 16028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.129:39265 every 8 connection(s)
I20260811 05:13:43.351285 15809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
I20260811 05:13:43.351697 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 15809
I20260811 05:13:43.351799 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance
I20260811 05:13:43.352013 29246 external_mini_cluster.cc:1428] Setting key 336af43fc6332dd53bf373a92369e6f5
Aug 11 05:13:43 dist-test-slave-f7th krb5kdc[15622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425223, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.28.143.130@KRBTEST.COM; defaulting to no policy
I20260811 05:13:43.362234 15771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:43.352109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:58255 (local address 127.28.143.190:41661)
0811 05:13:43.352163 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:43.352166 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:43.352484 (+ 318us) server_negotiation.cc:407] Connection header received
0811 05:13:43.353190 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:43.353191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:43.353206 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:43.353265 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:43.354570 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.355118 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.356279 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.356503 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.356567 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:43.359062 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:43.359064 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:43.359064 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:43.359075 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:43.360667 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:43.361189 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:43.361190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:43.361190 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:43.361249 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:43.361541 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:43.361541 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:43.361542 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:43.361651 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:43.361701 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:43.362067 (+ 366us) server_negotiation.cc:299] Negotiation successful
0811 05:13:43.362160 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 05:13:43.362458 16037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:43.352057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:41661 (local address 127.28.143.129:58255)
0811 05:13:43.352348 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:43.352387 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:43.353048 (+ 661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:43.353354 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:43.353359 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:43.354150 (+ 791us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:43.354437 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:43.354444 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.355231 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.355233 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:43.356179 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:43.356182 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.356594 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.356595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:43.356763 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:43.357302 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:43.357312 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:43.358927 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:43.360761 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:43.360770 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:43.360773 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:43.361106 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:43.361320 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:43.361321 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:43.361321 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:43.361472 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:43.361712 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:43.361720 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:43.361897 (+ 177us) client_negotiation.cc:769] Sending connection context
0811 05:13:43.362022 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 05:13:43.362222 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":99,"threads_started":1}
I20260811 05:13:43.363027 16030 heartbeater.cc:347] Connected to a master server at 127.28.143.190:41661
I20260811 05:13:43.363158 16030 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:43.363384 16030 heartbeater.cc:511] Master 127.28.143.190:41661 requested a full tablet report, sending...
I20260811 05:13:43.364034 15697 ts_manager.cc:194] Registered new tserver with Master: 12f8c822fb7a4fc58bede437ec1166a5 (127.28.143.129:39265)
I20260811 05:13:43.364775 15697 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.129@KRBTEST.COM'} at 127.28.143.129:58255
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab.
Entry for principal kudu/127.28.143.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab.
I20260811 05:13:43.384424 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.130:0
--local_ip_for_outbound_sockets=127.28.143.130
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:41661
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:35655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:43.477972 16043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:43.478235 16043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:43.478271 16043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:43.479696 16043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:43.479765 16043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:43.479806 16043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.130
I20260811 05:13:43.481680 16043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:35655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:41661
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.130
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:43.482021 16043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:43.482285 16043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:43.488709 16051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:43.488693 16052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:43.488698 16055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:43.489087 16043 server_base.cc:1034] running on GCE node
I20260811 05:13:43.489338 16043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:43.489614 16043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:43.490803 16043 hybrid_clock.cc:648] HybridClock initialized: now 1786425223490781 us; error 41 us; skew 500 ppm
Aug 11 05:13:43 dist-test-slave-f7th krb5kdc[15622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425223, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:43.493884 16043 init.cc:377] Logged in from keytab as kudu/127.28.143.130@KRBTEST.COM (short username kudu)
I20260811 05:13:43.494459 16043 webserver.cc:460] Webserver started at http://127.28.143.130:42635/ using document root <none> and password file <none>
I20260811 05:13:43.494675 16043 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:43.494720 16043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:43.494822 16043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:43.496119 16043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/data/instance:
uuid: "d5f78717245e4168b2eec5b36d39e664"
format_stamp: "Formatted at 2026-08-11 05:13:43 on dist-test-slave-f7th"
server_key: "511d958f6a226e1a8cebe24f89815d01"
server_key_iv: "7b36c60a12424bbcfdea8da26251b011"
server_key_version: "encryptionkey@0"
I20260811 05:13:43.496500 16043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance:
uuid: "d5f78717245e4168b2eec5b36d39e664"
format_stamp: "Formatted at 2026-08-11 05:13:43 on dist-test-slave-f7th"
server_key: "511d958f6a226e1a8cebe24f89815d01"
server_key_iv: "7b36c60a12424bbcfdea8da26251b011"
server_key_version: "encryptionkey@0"
I20260811 05:13:43.498284 16043 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260811 05:13:43.499210 16068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:43.499605 16043 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:43.499716 16043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/wal
uuid: "d5f78717245e4168b2eec5b36d39e664"
format_stamp: "Formatted at 2026-08-11 05:13:43 on dist-test-slave-f7th"
server_key: "511d958f6a226e1a8cebe24f89815d01"
server_key_iv: "7b36c60a12424bbcfdea8da26251b011"
server_key_version: "encryptionkey@0"
I20260811 05:13:43.499791 16043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:43.522953 16043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:43.525342 16043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:43.525478 16043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:43.525704 16043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:43.526043 16043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:43.526080 16043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:43.526103 16043 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:43.526126 16043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:43.532130 16043 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.130:34941
I20260811 05:13:43.532205 16285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.130:34941 every 8 connection(s)
I20260811 05:13:43.532549 16043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
Aug 11 05:13:43 dist-test-slave-f7th krb5kdc[15622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425223, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:13:43.541712 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 16043
I20260811 05:13:43.541828 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance
I20260811 05:13:43.542037 29246 external_mini_cluster.cc:1428] Setting key 7b37bfa540084430a6c1c865a3ab772b
I20260811 05:13:43.543259 15771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:43.533728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:46717 (local address 127.28.143.190:41661)
0811 05:13:43.533798 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:43.533800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:43.533814 (+ 14us) server_negotiation.cc:407] Connection header received
0811 05:13:43.534479 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:43.534480 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:43.534493 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:43.534557 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:43.535683 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.536249 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.537453 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.537659 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.537728 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:43.540045 (+ 2317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:43.540047 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:43.540048 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:43.540056 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:43.541425 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:43.542006 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:43.542007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:43.542008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:43.542073 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:43.542426 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:43.542427 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:43.542427 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:43.542603 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:43.542672 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:43.543097 (+ 425us) server_negotiation.cc:299] Negotiation successful
0811 05:13:43.543190 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 05:13:43.543509 16290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:43.533340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:41661 (local address 127.28.143.130:46717)
0811 05:13:43.533516 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:43.533590 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:43.534242 (+ 652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:43.534702 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:43.534707 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:43.535331 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:43.535558 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:43.535562 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.536389 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.536390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:43.537323 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:43.537326 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.537743 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.537744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:43.537854 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:43.538428 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:43.538437 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:43.539925 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:43.541539 (+ 1614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:43.541551 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:43.541554 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:43.541928 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:43.542199 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:43.542200 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:43.542200 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:43.542351 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:43.542697 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:43.542707 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:43.542902 (+ 195us) client_negotiation.cc:769] Sending connection context
0811 05:13:43.543073 (+ 171us) client_negotiation.cc:240] Negotiation successful
0811 05:13:43.543272 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":74,"threads_started":1}
I20260811 05:13:43.544091 16286 heartbeater.cc:347] Connected to a master server at 127.28.143.190:41661
I20260811 05:13:43.544205 16286 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:43.544409 16286 heartbeater.cc:511] Master 127.28.143.190:41661 requested a full tablet report, sending...
I20260811 05:13:43.545056 15697 ts_manager.cc:194] Registered new tserver with Master: d5f78717245e4168b2eec5b36d39e664 (127.28.143.130:34941)
I20260811 05:13:43.545468 15697 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.130@KRBTEST.COM'} at 127.28.143.130:46717
WARNING: no policy specified for kudu/127.28.143.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab.
Entry for principal kudu/127.28.143.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab.
I20260811 05:13:43.572779 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.131:0
--local_ip_for_outbound_sockets=127.28.143.131
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:41661
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:35655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:43.658097 16296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:43.658329 16296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:43.658354 16296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:43.659598 16296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:43.659657 16296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:43.659687 16296 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.131
I20260811 05:13:43.661432 16296 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:35655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:41661
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16296
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.131
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:13:43.661706 16296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:43.661962 16296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:43.667398 16302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:43.667392 16308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:43.667420 16296 server_base.cc:1034] running on GCE node
W20260811 05:13:43.667407 16305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:43.667783 16296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:43.668043 16296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:43.669189 16296 hybrid_clock.cc:648] HybridClock initialized: now 1786425223669170 us; error 39 us; skew 500 ppm
Aug 11 05:13:43 dist-test-slave-f7th krb5kdc[15622](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425223, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:43.671722 16296 init.cc:377] Logged in from keytab as kudu/127.28.143.131@KRBTEST.COM (short username kudu)
I20260811 05:13:43.672232 16296 webserver.cc:460] Webserver started at http://127.28.143.131:43443/ using document root <none> and password file <none>
I20260811 05:13:43.672451 16296 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:43.672513 16296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:43.672657 16296 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:43.673781 16296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/data/instance:
uuid: "66739311fd314cecb060ef4303988282"
format_stamp: "Formatted at 2026-08-11 05:13:43 on dist-test-slave-f7th"
server_key: "9b59e2c6c22e260c563251eaa1c41bd0"
server_key_iv: "bb4396d7ad9e8d9694c451cd97cea7ce"
server_key_version: "encryptionkey@0"
I20260811 05:13:43.674147 16296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance:
uuid: "66739311fd314cecb060ef4303988282"
format_stamp: "Formatted at 2026-08-11 05:13:43 on dist-test-slave-f7th"
server_key: "9b59e2c6c22e260c563251eaa1c41bd0"
server_key_iv: "bb4396d7ad9e8d9694c451cd97cea7ce"
server_key_version: "encryptionkey@0"
I20260811 05:13:43.675578 16296 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 05:13:43.676425 16320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:43.676700 16296 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 05:13:43.676808 16296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/wal
uuid: "66739311fd314cecb060ef4303988282"
format_stamp: "Formatted at 2026-08-11 05:13:43 on dist-test-slave-f7th"
server_key: "9b59e2c6c22e260c563251eaa1c41bd0"
server_key_iv: "bb4396d7ad9e8d9694c451cd97cea7ce"
server_key_version: "encryptionkey@0"
I20260811 05:13:43.676879 16296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:43.690527 16296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:43.692837 16296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:43.692998 16296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:43.693236 16296 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:43.693547 16296 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:43.693583 16296 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:43.693614 16296 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:43.693637 16296 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:43.699304 16296 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.131:46763
I20260811 05:13:43.699380 16500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.131:46763 every 8 connection(s)
I20260811 05:13:43.699694 16296 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
Aug 11 05:13:43 dist-test-slave-f7th krb5kdc[15622](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425223, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:13:43.709317 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 16296
I20260811 05:13:43.709429 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance
I20260811 05:13:43.709590 29246 external_mini_cluster.cc:1428] Setting key b173c8ece8040c267c187bc08bee31fa
I20260811 05:13:43.709781 15771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:43.700479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:47071 (local address 127.28.143.190:41661)
0811 05:13:43.700559 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:43.700561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:43.700897 (+ 336us) server_negotiation.cc:407] Connection header received
0811 05:13:43.701350 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:43.701373 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:43.701385 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:43.701445 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:43.702502 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.703092 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.704111 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.704304 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.704370 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:43.706760 (+ 2390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:43.706762 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:43.706762 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:43.706771 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:43.708228 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:43.708760 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:43.708762 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:43.708762 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:43.708827 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:43.709100 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:43.709101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:43.709102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:43.709220 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:43.709276 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:43.709595 (+ 319us) server_negotiation.cc:299] Negotiation successful
0811 05:13:43.709688 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 05:13:43.709975 16505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:43.700418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:41661 (local address 127.28.143.131:47071)
0811 05:13:43.700658 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:43.700710 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:43.701230 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:43.701535 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:43.701539 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:43.702197 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:43.702410 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:43.702415 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.703176 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.703177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:43.703997 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:43.704000 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.704432 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.704433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:43.704553 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:43.705159 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:43.705169 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:43.706646 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:43.708340 (+ 1694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:43.708380 (+ 40us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:43.708383 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:43.708682 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:43.708904 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:43.708905 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:43.708905 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:43.709025 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:43.709286 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:43.709294 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:43.709448 (+ 154us) client_negotiation.cc:769] Sending connection context
0811 05:13:43.709557 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 05:13:43.709733 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":90,"threads_started":1}
I20260811 05:13:43.710551 16501 heartbeater.cc:347] Connected to a master server at 127.28.143.190:41661
I20260811 05:13:43.710657 16501 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:43.710848 16501 heartbeater.cc:511] Master 127.28.143.190:41661 requested a full tablet report, sending...
I20260811 05:13:43.711264 15697 ts_manager.cc:194] Registered new tserver with Master: 66739311fd314cecb060ef4303988282 (127.28.143.131:46763)
I20260811 05:13:43.711694 15697 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.131@KRBTEST.COM'} at 127.28.143.131:47071
I20260811 05:13:43.721643 29246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:13:43.729475 15771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:43.722978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35450 (local address 127.28.143.190:41661)
0811 05:13:43.723039 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:43.723041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:43.723133 (+ 92us) server_negotiation.cc:407] Connection header received
0811 05:13:43.723175 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:43.723177 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:43.723185 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:43.723231 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:43.723768 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.724279 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.725151 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.725356 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.725426 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:43.726719 (+ 1293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:43.726720 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:43.726720 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:43.726728 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:43.728201 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:43.728575 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:43.728575 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:43.728576 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:43.728690 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:43.728939 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:43.728940 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:43.728941 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:43.729111 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:43.729194 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:43.729300 (+ 106us) server_negotiation.cc:299] Negotiation successful
0811 05:13:43.729382 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 05:13:43.731524 15697 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35450:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:13:43.733297 15697 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:13:43.740355 16527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:43.736176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45804 (local address 127.28.143.130:34941)
0811 05:13:43.736333 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:43.736337 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:43.736348 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:13:43.736397 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:43.736399 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:43.736414 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:43.736492 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:43.736693 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.737793 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.739099 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.739814 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.739901 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:43.739965 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:43.740217 (+ 252us) server_negotiation.cc:299] Negotiation successful
0811 05:13:43.740264 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":70,"threads_started":1}
I20260811 05:13:43.740356 16518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:43.735681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:34941 (local address 127.0.0.1:45804)
0811 05:13:43.736191 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:43.736200 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:43.736234 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:43.736569 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:43.736570 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:43.736571 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:43.736605 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:43.736607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.737944 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.737946 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:43.739033 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:43.739035 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.739943 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.739945 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:43.740065 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:43.740087 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 05:13:43.740176 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 05:13:43.740245 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"mutex_wait_us":5,"thread_start_us":87,"threads_started":1}
I20260811 05:13:43.740444 16516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:43.735685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:39265 (local address 127.0.0.1:44056)
0811 05:13:43.735970 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:43.736004 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:43.736062 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:43.736548 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:43.736549 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:43.736551 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:43.736642 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:43.736644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.737568 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.737569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:43.738931 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:43.738935 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.739945 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.739946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:43.740070 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:43.740087 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 05:13:43.740176 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 05:13:43.740245 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"mutex_wait_us":28,"thread_start_us":99,"threads_started":1}
I20260811 05:13:43.740818 16519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:43.735789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44056 (local address 127.28.143.129:39265)
0811 05:13:43.736365 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:43.736369 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:43.736378 (+ 9us) server_negotiation.cc:407] Connection header received
0811 05:13:43.736433 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:43.736436 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:43.736451 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:43.736712 (+ 261us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:43.736784 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.737468 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.739036 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.739815 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.739901 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:43.739965 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:43.740681 (+ 716us) server_negotiation.cc:299] Negotiation successful
0811 05:13:43.740736 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":95,"threads_started":1}
I20260811 05:13:43.741400 16156 tablet_service.cc:1467] Processing CreateTablet for tablet b45d78ff66c94d1e883454b36428cea7 (DEFAULT_TABLE table=test-table [id=eafdaaf3aee54f27a45c85757ab640a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:43.741564 16517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:43.735680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.131:46763 (local address 127.0.0.1:36858)
0811 05:13:43.735963 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:43.736003 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:43.736062 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:43.736356 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:43.736359 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:43.736360 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:43.736436 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:43.736438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.738017 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.738019 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:43.738944 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:43.738946 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.741347 (+ 2401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.741348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:43.741416 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:43.741421 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 05:13:43.741451 (+ 30us) client_negotiation.cc:240] Negotiation successful
0811 05:13:43.741495 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":66,"threads_started":1}
I20260811 05:13:43.741622 15911 tablet_service.cc:1467] Processing CreateTablet for tablet b45d78ff66c94d1e883454b36428cea7 (DEFAULT_TABLE table=test-table [id=eafdaaf3aee54f27a45c85757ab640a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:43.741778 16156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b45d78ff66c94d1e883454b36428cea7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:43.741971 16523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:43.735970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36858 (local address 127.28.143.131:46763)
0811 05:13:43.736172 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:43.736178 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:43.736188 (+ 10us) server_negotiation.cc:407] Connection header received
0811 05:13:43.736232 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:43.736234 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:43.736251 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:43.736340 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:43.736516 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.737930 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.740012 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.741249 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.741749 (+ 500us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:43.741817 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:43.741845 (+ 28us) server_negotiation.cc:299] Negotiation successful
0811 05:13:43.741891 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":97,"threads_started":1}
I20260811 05:13:43.741991 15911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b45d78ff66c94d1e883454b36428cea7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:43.742663 16402 tablet_service.cc:1467] Processing CreateTablet for tablet b45d78ff66c94d1e883454b36428cea7 (DEFAULT_TABLE table=test-table [id=eafdaaf3aee54f27a45c85757ab640a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:43.743075 16402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b45d78ff66c94d1e883454b36428cea7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:43.745200 16530 tablet_bootstrap.cc:492] T b45d78ff66c94d1e883454b36428cea7 P d5f78717245e4168b2eec5b36d39e664: Bootstrap starting.
I20260811 05:13:43.745296 16528 tablet_bootstrap.cc:492] T b45d78ff66c94d1e883454b36428cea7 P 12f8c822fb7a4fc58bede437ec1166a5: Bootstrap starting.
I20260811 05:13:43.745887 16530 tablet_bootstrap.cc:654] T b45d78ff66c94d1e883454b36428cea7 P d5f78717245e4168b2eec5b36d39e664: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:43.746093 16528 tablet_bootstrap.cc:654] T b45d78ff66c94d1e883454b36428cea7 P 12f8c822fb7a4fc58bede437ec1166a5: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:43.746415 16530 log.cc:824] T b45d78ff66c94d1e883454b36428cea7 P d5f78717245e4168b2eec5b36d39e664: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:43.746500 16533 tablet_bootstrap.cc:492] T b45d78ff66c94d1e883454b36428cea7 P 66739311fd314cecb060ef4303988282: Bootstrap starting.
I20260811 05:13:43.746662 16528 log.cc:824] T b45d78ff66c94d1e883454b36428cea7 P 12f8c822fb7a4fc58bede437ec1166a5: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:43.747201 16533 tablet_bootstrap.cc:654] T b45d78ff66c94d1e883454b36428cea7 P 66739311fd314cecb060ef4303988282: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:43.747567 16528 tablet_bootstrap.cc:492] T b45d78ff66c94d1e883454b36428cea7 P 12f8c822fb7a4fc58bede437ec1166a5: No bootstrap required, opened a new log
I20260811 05:13:43.747638 16528 ts_tablet_manager.cc:1397] T b45d78ff66c94d1e883454b36428cea7 P 12f8c822fb7a4fc58bede437ec1166a5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 05:13:43.747759 16533 log.cc:824] T b45d78ff66c94d1e883454b36428cea7 P 66739311fd314cecb060ef4303988282: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:43.748481 16533 tablet_bootstrap.cc:492] T b45d78ff66c94d1e883454b36428cea7 P 66739311fd314cecb060ef4303988282: No bootstrap required, opened a new log
I20260811 05:13:43.748529 16533 ts_tablet_manager.cc:1397] T b45d78ff66c94d1e883454b36428cea7 P 66739311fd314cecb060ef4303988282: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 05:13:43.749269 16528 raft_consensus.cc:348] T b45d78ff66c94d1e883454b36428cea7 P 12f8c822fb7a4fc58bede437ec1166a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66739311fd314cecb060ef4303988282" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 46763 } } peers { permanent_uuid: "12f8c822fb7a4fc58bede437ec1166a5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 39265 } } peers { permanent_uuid: "d5f78717245e4168b2eec5b36d39e664" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34941 } }
I20260811 05:13:43.749425 16528 raft_consensus.cc:374] T b45d78ff66c94d1e883454b36428cea7 P 12f8c822fb7a4fc58bede437ec1166a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:43.749449 16528 raft_consensus.cc:729] T b45d78ff66c94d1e883454b36428cea7 P 12f8c822fb7a4fc58bede437ec1166a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12f8c822fb7a4fc58bede437ec1166a5, State: Initialized, Role: FOLLOWER
I20260811 05:13:43.749541 16528 consensus_queue.cc:260] T b45d78ff66c94d1e883454b36428cea7 P 12f8c822fb7a4fc58bede437ec1166a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66739311fd314cecb060ef4303988282" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 46763 } } peers { permanent_uuid: "12f8c822fb7a4fc58bede437ec1166a5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 39265 } } peers { permanent_uuid: "d5f78717245e4168b2eec5b36d39e664" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34941 } }
I20260811 05:13:43.749804 16528 ts_tablet_manager.cc:1428] T b45d78ff66c94d1e883454b36428cea7 P 12f8c822fb7a4fc58bede437ec1166a5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 05:13:43.749886 16030 heartbeater.cc:503] Master 127.28.143.190:41661 was elected leader, sending a full tablet report...
I20260811 05:13:43.750195 16533 raft_consensus.cc:348] T b45d78ff66c94d1e883454b36428cea7 P 66739311fd314cecb060ef4303988282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66739311fd314cecb060ef4303988282" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 46763 } } peers { permanent_uuid: "12f8c822fb7a4fc58bede437ec1166a5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 39265 } } peers { permanent_uuid: "d5f78717245e4168b2eec5b36d39e664" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34941 } }
I20260811 05:13:43.750361 16533 raft_consensus.cc:374] T b45d78ff66c94d1e883454b36428cea7 P 66739311fd314cecb060ef4303988282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:43.750396 16533 raft_consensus.cc:729] T b45d78ff66c94d1e883454b36428cea7 P 66739311fd314cecb060ef4303988282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66739311fd314cecb060ef4303988282, State: Initialized, Role: FOLLOWER
I20260811 05:13:43.750567 16530 tablet_bootstrap.cc:492] T b45d78ff66c94d1e883454b36428cea7 P d5f78717245e4168b2eec5b36d39e664: No bootstrap required, opened a new log
I20260811 05:13:43.750552 16533 consensus_queue.cc:260] T b45d78ff66c94d1e883454b36428cea7 P 66739311fd314cecb060ef4303988282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66739311fd314cecb060ef4303988282" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 46763 } } peers { permanent_uuid: "12f8c822fb7a4fc58bede437ec1166a5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 39265 } } peers { permanent_uuid: "d5f78717245e4168b2eec5b36d39e664" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34941 } }
I20260811 05:13:43.750699 16530 ts_tablet_manager.cc:1397] T b45d78ff66c94d1e883454b36428cea7 P d5f78717245e4168b2eec5b36d39e664: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260811 05:13:43.750839 16533 ts_tablet_manager.cc:1428] T b45d78ff66c94d1e883454b36428cea7 P 66739311fd314cecb060ef4303988282: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 05:13:43.750890 16501 heartbeater.cc:503] Master 127.28.143.190:41661 was elected leader, sending a full tablet report...
I20260811 05:13:43.752466 16530 raft_consensus.cc:348] T b45d78ff66c94d1e883454b36428cea7 P d5f78717245e4168b2eec5b36d39e664 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66739311fd314cecb060ef4303988282" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 46763 } } peers { permanent_uuid: "12f8c822fb7a4fc58bede437ec1166a5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 39265 } } peers { permanent_uuid: "d5f78717245e4168b2eec5b36d39e664" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34941 } }
I20260811 05:13:43.752656 16530 raft_consensus.cc:374] T b45d78ff66c94d1e883454b36428cea7 P d5f78717245e4168b2eec5b36d39e664 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:43.752700 16530 raft_consensus.cc:729] T b45d78ff66c94d1e883454b36428cea7 P d5f78717245e4168b2eec5b36d39e664 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5f78717245e4168b2eec5b36d39e664, State: Initialized, Role: FOLLOWER
I20260811 05:13:43.752815 16530 consensus_queue.cc:260] T b45d78ff66c94d1e883454b36428cea7 P d5f78717245e4168b2eec5b36d39e664 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66739311fd314cecb060ef4303988282" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 46763 } } peers { permanent_uuid: "12f8c822fb7a4fc58bede437ec1166a5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 39265 } } peers { permanent_uuid: "d5f78717245e4168b2eec5b36d39e664" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34941 } }
I20260811 05:13:43.753062 16530 ts_tablet_manager.cc:1428] T b45d78ff66c94d1e883454b36428cea7 P d5f78717245e4168b2eec5b36d39e664: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 05:13:43.753129 16286 heartbeater.cc:503] Master 127.28.143.190:41661 was elected leader, sending a full tablet report...
W20260811 05:13:43.783188 16288 tablet.cc:2369] T b45d78ff66c94d1e883454b36428cea7 P d5f78717245e4168b2eec5b36d39e664: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 05:13:43.852178 16033 tablet.cc:2369] T b45d78ff66c94d1e883454b36428cea7 P 12f8c822fb7a4fc58bede437ec1166a5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 05:13:43.950326 16502 tablet.cc:2369] T b45d78ff66c94d1e883454b36428cea7 P 66739311fd314cecb060ef4303988282: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:13:43.997043 16543 raft_consensus.cc:482] T b45d78ff66c94d1e883454b36428cea7 P 66739311fd314cecb060ef4303988282 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:13:43.997197 16543 raft_consensus.cc:504] T b45d78ff66c94d1e883454b36428cea7 P 66739311fd314cecb060ef4303988282 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66739311fd314cecb060ef4303988282" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 46763 } } peers { permanent_uuid: "12f8c822fb7a4fc58bede437ec1166a5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 39265 } } peers { permanent_uuid: "d5f78717245e4168b2eec5b36d39e664" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34941 } }
I20260811 05:13:43.997619 16543 leader_election.cc:288] T b45d78ff66c94d1e883454b36428cea7 P 66739311fd314cecb060ef4303988282 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12f8c822fb7a4fc58bede437ec1166a5 (127.28.143.129:39265), d5f78717245e4168b2eec5b36d39e664 (127.28.143.130:34941)
I20260811 05:13:44.000669 16519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:43.997938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:59073 (local address 127.28.143.129:39265)
0811 05:13:43.997983 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:43.997985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:43.998141 (+ 156us) server_negotiation.cc:407] Connection header received
0811 05:13:43.998189 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:43.998190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:43.998200 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:43.998250 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:43.998430 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.998875 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.999936 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:44.000293 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:44.000353 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:44.000362 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:44.000530 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 05:13:44.000570 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 05:13:44.000684 16505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:43.997834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:34941 (local address 127.28.143.131:36315)
0811 05:13:43.997977 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:43.997991 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:43.998052 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:43.998189 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:43.998190 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:43.998192 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:43.998276 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:43.998277 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.998975 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.998976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:43.999843 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:43.999845 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:44.000396 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:44.000397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:44.000478 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:44.000481 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 05:13:44.000534 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 05:13:44.000572 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":50}
I20260811 05:13:44.000680 16527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:43.997951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:36315 (local address 127.28.143.130:34941)
0811 05:13:43.998008 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:43.998010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:43.998051 (+ 41us) server_negotiation.cc:407] Connection header received
0811 05:13:43.998110 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:43.998111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:43.998122 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:43.998176 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:43.998346 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.998848 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.999944 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:44.000312 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:44.000381 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:44.000390 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:44.000553 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 05:13:44.000595 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 05:13:44.000685 16545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:43.997858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:39265 (local address 127.28.143.131:59073)
0811 05:13:43.998053 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:43.998063 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:43.998130 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:43.998272 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:43.998273 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:43.998274 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:43.998343 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:43.998345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.998966 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.998967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:43.999840 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:43.999842 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:44.000383 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:44.000384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:44.000458 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:44.000463 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 05:13:44.000506 (+ 43us) client_negotiation.cc:240] Negotiation successful
0811 05:13:44.000558 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":26,"thread_start_us":117,"threads_started":1}
I20260811 05:13:44.001055 16197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b45d78ff66c94d1e883454b36428cea7" candidate_uuid: "66739311fd314cecb060ef4303988282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5f78717245e4168b2eec5b36d39e664" is_pre_election: true
I20260811 05:13:44.001067 15947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b45d78ff66c94d1e883454b36428cea7" candidate_uuid: "66739311fd314cecb060ef4303988282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12f8c822fb7a4fc58bede437ec1166a5" is_pre_election: true
I20260811 05:13:44.001230 15947 raft_consensus.cc:2454] T b45d78ff66c94d1e883454b36428cea7 P 12f8c822fb7a4fc58bede437ec1166a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66739311fd314cecb060ef4303988282 in term 0.
I20260811 05:13:44.001230 16197 raft_consensus.cc:2454] T b45d78ff66c94d1e883454b36428cea7 P d5f78717245e4168b2eec5b36d39e664 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66739311fd314cecb060ef4303988282 in term 0.
I20260811 05:13:44.001472 16326 leader_election.cc:302] T b45d78ff66c94d1e883454b36428cea7 P 66739311fd314cecb060ef4303988282 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66739311fd314cecb060ef4303988282, d5f78717245e4168b2eec5b36d39e664; no voters:
I20260811 05:13:44.001597 16543 raft_consensus.cc:2781] T b45d78ff66c94d1e883454b36428cea7 P 66739311fd314cecb060ef4303988282 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:13:44.001674 16543 raft_consensus.cc:482] T b45d78ff66c94d1e883454b36428cea7 P 66739311fd314cecb060ef4303988282 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:13:44.001693 16543 raft_consensus.cc:3037] T b45d78ff66c94d1e883454b36428cea7 P 66739311fd314cecb060ef4303988282 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:44.002457 16543 raft_consensus.cc:504] T b45d78ff66c94d1e883454b36428cea7 P 66739311fd314cecb060ef4303988282 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66739311fd314cecb060ef4303988282" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 46763 } } peers { permanent_uuid: "12f8c822fb7a4fc58bede437ec1166a5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 39265 } } peers { permanent_uuid: "d5f78717245e4168b2eec5b36d39e664" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34941 } }
I20260811 05:13:44.002620 16543 leader_election.cc:288] T b45d78ff66c94d1e883454b36428cea7 P 66739311fd314cecb060ef4303988282 [CANDIDATE]: Term 1 election: Requested vote from peers 12f8c822fb7a4fc58bede437ec1166a5 (127.28.143.129:39265), d5f78717245e4168b2eec5b36d39e664 (127.28.143.130:34941)
I20260811 05:13:44.002806 15947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b45d78ff66c94d1e883454b36428cea7" candidate_uuid: "66739311fd314cecb060ef4303988282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12f8c822fb7a4fc58bede437ec1166a5"
I20260811 05:13:44.002903 15947 raft_consensus.cc:3037] T b45d78ff66c94d1e883454b36428cea7 P 12f8c822fb7a4fc58bede437ec1166a5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:44.002978 16197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b45d78ff66c94d1e883454b36428cea7" candidate_uuid: "66739311fd314cecb060ef4303988282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5f78717245e4168b2eec5b36d39e664"
I20260811 05:13:44.003082 16197 raft_consensus.cc:3037] T b45d78ff66c94d1e883454b36428cea7 P d5f78717245e4168b2eec5b36d39e664 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:44.003736 15947 raft_consensus.cc:2454] T b45d78ff66c94d1e883454b36428cea7 P 12f8c822fb7a4fc58bede437ec1166a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66739311fd314cecb060ef4303988282 in term 1.
I20260811 05:13:44.003851 16197 raft_consensus.cc:2454] T b45d78ff66c94d1e883454b36428cea7 P d5f78717245e4168b2eec5b36d39e664 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66739311fd314cecb060ef4303988282 in term 1.
I20260811 05:13:44.003947 16323 leader_election.cc:302] T b45d78ff66c94d1e883454b36428cea7 P 66739311fd314cecb060ef4303988282 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12f8c822fb7a4fc58bede437ec1166a5, 66739311fd314cecb060ef4303988282; no voters:
I20260811 05:13:44.004046 16543 raft_consensus.cc:2781] T b45d78ff66c94d1e883454b36428cea7 P 66739311fd314cecb060ef4303988282 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:44.004202 16543 raft_consensus.cc:686] T b45d78ff66c94d1e883454b36428cea7 P 66739311fd314cecb060ef4303988282 [term 1 LEADER]: Becoming Leader. State: Replica: 66739311fd314cecb060ef4303988282, State: Running, Role: LEADER
I20260811 05:13:44.004299 16543 consensus_queue.cc:237] T b45d78ff66c94d1e883454b36428cea7 P 66739311fd314cecb060ef4303988282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66739311fd314cecb060ef4303988282" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 46763 } } peers { permanent_uuid: "12f8c822fb7a4fc58bede437ec1166a5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 39265 } } peers { permanent_uuid: "d5f78717245e4168b2eec5b36d39e664" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34941 } }
I20260811 05:13:44.005331 15697 catalog_manager.cc:5495] T b45d78ff66c94d1e883454b36428cea7 P 66739311fd314cecb060ef4303988282 reported cstate change: term changed from 0 to 1, leader changed from <none> to 66739311fd314cecb060ef4303988282 (127.28.143.131). New cstate: current_term: 1 leader_uuid: "66739311fd314cecb060ef4303988282" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66739311fd314cecb060ef4303988282" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 46763 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12f8c822fb7a4fc58bede437ec1166a5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 39265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5f78717245e4168b2eec5b36d39e664" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34941 } health_report { overall_health: UNKNOWN } } }
I20260811 05:13:44.025493 16523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:44.022865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36870 (local address 127.28.143.131:46763)
0811 05:13:44.022970 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:44.022972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:44.022983 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:13:44.023021 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:44.023022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:44.023042 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:44.023110 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:13:44.023287 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:44.023793 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:44.024665 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:44.024939 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:44.025002 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:44.025186 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:13:44.025301 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:13:44.025376 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:44.025389 (+ 13us) server_negotiation.cc:299] Negotiation successful
0811 05:13:44.025409 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 05:13:44.027477 16197 raft_consensus.cc:1261] T b45d78ff66c94d1e883454b36428cea7 P d5f78717245e4168b2eec5b36d39e664 [term 1 FOLLOWER]: Refusing update from remote peer 66739311fd314cecb060ef4303988282: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:44.027465 15947 raft_consensus.cc:1261] T b45d78ff66c94d1e883454b36428cea7 P 12f8c822fb7a4fc58bede437ec1166a5 [term 1 FOLLOWER]: Refusing update from remote peer 66739311fd314cecb060ef4303988282: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:44.027845 16546 consensus_queue.cc:1035] T b45d78ff66c94d1e883454b36428cea7 P 66739311fd314cecb060ef4303988282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12f8c822fb7a4fc58bede437ec1166a5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 39265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:44.027980 16543 consensus_queue.cc:1035] T b45d78ff66c94d1e883454b36428cea7 P 66739311fd314cecb060ef4303988282 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5f78717245e4168b2eec5b36d39e664" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 34941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:44.031225 16556 mvcc.cc:205] Tried to move back new op lower bound from 7317197717613600768 to 7317197717522350080. Current Snapshot: MvccSnapshot[applied={T|T < 7317197717613600768}]
I20260811 05:13:44.032639 16561 mvcc.cc:205] Tried to move back new op lower bound from 7317197717613600768 to 7317197717522350080. Current Snapshot: MvccSnapshot[applied={T|T < 7317197717613600768 or (T in {7317197717613600768})}]
I20260811 05:13:44.032883 15695 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35450:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:44.032986 15695 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35450:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:44.033783 15695 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8477996b1b214cde87d4faef02ecffaf: Sending DeleteTablet for 3 replicas of tablet b45d78ff66c94d1e883454b36428cea7
I20260811 05:13:44.034160 15911 tablet_service.cc:1514] Processing DeleteTablet for tablet b45d78ff66c94d1e883454b36428cea7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:44 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:44056
I20260811 05:13:44.034178 16156 tablet_service.cc:1514] Processing DeleteTablet for tablet b45d78ff66c94d1e883454b36428cea7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:44 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:45804
I20260811 05:13:44.034219 16402 tablet_service.cc:1514] Processing DeleteTablet for tablet b45d78ff66c94d1e883454b36428cea7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:44 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:36858
I20260811 05:13:44.034439 16584 tablet_replica.cc:330] T b45d78ff66c94d1e883454b36428cea7 P d5f78717245e4168b2eec5b36d39e664: stopping tablet replica
I20260811 05:13:44.034570 16581 tablet_replica.cc:330] T b45d78ff66c94d1e883454b36428cea7 P 12f8c822fb7a4fc58bede437ec1166a5: stopping tablet replica
I20260811 05:13:44.034574 16584 raft_consensus.cc:2229] T b45d78ff66c94d1e883454b36428cea7 P d5f78717245e4168b2eec5b36d39e664 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:44.034698 16584 raft_consensus.cc:2258] T b45d78ff66c94d1e883454b36428cea7 P d5f78717245e4168b2eec5b36d39e664 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:44.034731 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 15809
I20260811 05:13:44.034843 16582 tablet_replica.cc:330] T b45d78ff66c94d1e883454b36428cea7 P 66739311fd314cecb060ef4303988282: stopping tablet replica
I20260811 05:13:44.034935 16582 raft_consensus.cc:2229] T b45d78ff66c94d1e883454b36428cea7 P 66739311fd314cecb060ef4303988282 [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:13:44.035136 16582 raft_consensus.cc:2258] T b45d78ff66c94d1e883454b36428cea7 P 66739311fd314cecb060ef4303988282 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:44.040591 16582 ts_tablet_manager.cc:1905] T b45d78ff66c94d1e883454b36428cea7 P 66739311fd314cecb060ef4303988282: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:44.042793 16582 ts_tablet_manager.cc:1918] T b45d78ff66c94d1e883454b36428cea7 P 66739311fd314cecb060ef4303988282: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:13:44.042881 16582 log.cc:1196] T b45d78ff66c94d1e883454b36428cea7 P 66739311fd314cecb060ef4303988282: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-2/wal/wals/b45d78ff66c94d1e883454b36428cea7
I20260811 05:13:44.043109 16582 ts_tablet_manager.cc:1939] T b45d78ff66c94d1e883454b36428cea7 P 66739311fd314cecb060ef4303988282: Deleting consensus metadata
I20260811 05:13:44.045478 15672 catalog_manager.cc:4841] TS 66739311fd314cecb060ef4303988282 (127.28.143.131:46763): tablet b45d78ff66c94d1e883454b36428cea7 (table test-table [id=eafdaaf3aee54f27a45c85757ab640a1]) successfully deleted
I20260811 05:13:44.051928 16584 ts_tablet_manager.cc:1905] T b45d78ff66c94d1e883454b36428cea7 P d5f78717245e4168b2eec5b36d39e664: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:44.053985 16584 ts_tablet_manager.cc:1918] T b45d78ff66c94d1e883454b36428cea7 P d5f78717245e4168b2eec5b36d39e664: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:13:44.054088 16584 log.cc:1196] T b45d78ff66c94d1e883454b36428cea7 P d5f78717245e4168b2eec5b36d39e664: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425159060450-29246-0/minicluster-data/ts-1/wal/wals/b45d78ff66c94d1e883454b36428cea7
I20260811 05:13:44.054354 16584 ts_tablet_manager.cc:1939] T b45d78ff66c94d1e883454b36428cea7 P d5f78717245e4168b2eec5b36d39e664: Deleting consensus metadata
I20260811 05:13:44.054911 15674 catalog_manager.cc:4841] TS d5f78717245e4168b2eec5b36d39e664 (127.28.143.130:34941): tablet b45d78ff66c94d1e883454b36428cea7 (table test-table [id=eafdaaf3aee54f27a45c85757ab640a1]) successfully deleted
W20260811 05:13:44.056988 15673 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.143.129:39265 (error 108)
I20260811 05:13:44.057250 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 16043
I20260811 05:13:44.085053 16516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:44.084880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:39265 (local address 127.0.0.1:44062)
0811 05:13:44.084945 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:44.085004 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.129:39265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 05:13:44.085237 15673 catalog_manager.cc:4584] TS 12f8c822fb7a4fc58bede437ec1166a5 (127.28.143.129:39265): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b45d78ff66c94d1e883454b36428cea7: Network error: Client connection negotiation failed: client connection to 127.28.143.129:39265: connect: Connection refused (error 111)
I20260811 05:13:44.117319 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 16296
I20260811 05:13:44.133455 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 15646
2026-08-11T05:13:44Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2517 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:13:44 dist-test-slave-f7th krb5kdc[16590](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:13:44 dist-test-slave-f7th krb5kdc[16590](info): set up 2 sockets
Aug 11 05:13:44 dist-test-slave-f7th krb5kdc[16590](info): commencing operation
krb5kdc: starting...
W20260811 05:13:45.401410 29246 mini_kdc.cc:121] Time spent starting KDC: real 1.228s user 0.000s sys 0.011s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Aug 11 05:13:45 dist-test-slave-f7th krb5kdc[16590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425225, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:13:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:13:45Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.28.143.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.28.143.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.28.143.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/krb5kdc/rangeradmin_127.28.143.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.28.143.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/krb5kdc/rangeradmin_127.28.143.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.28.143.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.28.143.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.28.143.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/krb5kdc/rangerlookup_127.28.143.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.28.143.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/krb5kdc/rangerlookup_127.28.143.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.28.143.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.28.143.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.28.143.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/krb5kdc/HTTP_127.28.143.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.28.143.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/krb5kdc/HTTP_127.28.143.148@KRBTEST.COM.keytab.
I20260811 05:13:45.636962 29246 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskY_TM7_/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/postgres -l logfile start
2026-08-11 05:13:48.904 UTC [16641] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 05:13:48.904 UTC [16641] LOG: listening on IPv4 address "127.28.143.148", port 40095
2026-08-11 05:13:48.909 UTC [16641] LOG: listening on Unix socket "/tmp/.s.PGSQL.40095"
2026-08-11 05:13:48.943 UTC [16645] LOG: database system was shut down at 2026-08-11 05:13:46 UTC
2026-08-11 05:13:48.947 UTC [16641] LOG: database system is ready to accept connections
I20260811 05:13:50.089062 29246 mini_postgres.cc:96] Postgres bound to 40095
2026-08-11 05:13:50.096 UTC [16660] FATAL: database "slave" does not exist
127.28.143.148:40095 - accepting connections
I20260811 05:13:50.097604 29246 mini_ranger.cc:161] Starting Ranger...
I20260811 05:13:50.114806 29246 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 05:13:50.755182 29246 mini_ranger.cc:87] Created ranger Postgres database
I20260811 05:13:50.755278 29246 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ranger-admin
2026-08-11 05:13:51,080 [I] DB FLAVOR :POSTGRES
2026-08-11 05:13:51,080 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 05:13:51,080 [I] Checking connection..
2026-08-11 05:13:51,080 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:40095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 05:13:51,379 [I] Checking connection passed.
2026-08-11 05:13:51,379 [I] --------- Verifying version history table ---------
2026-08-11 05:13:51,379 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:40095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 05:13:51,672 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 05:13:51,672 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:40095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 05:13:51,973 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 05:13:51,974 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 05:13:51,974 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:40095/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 05:13:52.253 UTC [16774] WARNING: there is no transaction in progress
2026-08-11 05:13:52,263 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 05:13:52,264 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:40095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 05:13:52,540 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 05:13:52,540 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 05:13:52,541 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:40095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 05:13:52,819 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:40095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 05:13:53,116 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:40095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-f7th.c.gcp-upstream.internal','N') ;"
2026-08-11 05:13:53,396 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:40095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 05:13:53,684 [I] Table x_portal_user does not exist in database ranger
2026-08-11 05:13:53,685 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:40095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 05:13:53,974 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 05:13:53,975 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:40095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 05:13:54,245 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:40095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 05:13:54,518 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 05:13:54,518 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:40095/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 05:13:55.852 UTC [16955] WARNING: there is no transaction in progress
2026-08-11 05:13:55.875 UTC [16955] WARNING: there is no transaction in progress
2026-08-11 05:13:55.898 UTC [16955] WARNING: there is no transaction in progress
2026-08-11 05:13:55.921 UTC [16955] WARNING: there is no transaction in progress
2026-08-11 05:13:55.947 UTC [16955] WARNING: there is no transaction in progress
2026-08-11 05:13:55.974 UTC [16955] WARNING: there is no transaction in progress
2026-08-11 05:13:56.056 UTC [16955] WARNING: there is no transaction in progress
2026-08-11 05:13:56.066 UTC [16955] WARNING: there is no transaction in progress
2026-08-11 05:13:56.089 UTC [16955] WARNING: there is no transaction in progress
2026-08-11 05:13:56.104 UTC [16955] WARNING: there is no transaction in progress
2026-08-11 05:13:56.118 UTC [16955] WARNING: there is no transaction in progress
2026-08-11 05:13:56.138 UTC [16955] WARNING: there is no transaction in progress
2026-08-11 05:13:56.152 UTC [16955] WARNING: there is no transaction in progress
2026-08-11 05:13:56.160 UTC [16955] WARNING: there is no transaction in progress
2026-08-11 05:13:56.169 UTC [16955] WARNING: there is no transaction in progress
2026-08-11 05:13:56.178 UTC [16955] WARNING: there is no transaction in progress
2026-08-11 05:13:57,139 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 05:13:57,139 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:40095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 05:13:57,431 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 05:13:57,431 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:40095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 05:13:57,745 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 05:13:57,746 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:40095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 05:13:58,085 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 05:13:58,085 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:40095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 05:13:58,365 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:40095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 05:13:58,637 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:40095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-f7th.c.gcp-upstream.internal';"
2026-08-11 05:13:58,940 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 05:13:58,940 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:40095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 05:13:59,261 [I] DB_PATCHES have already been applied
I20260811 05:13:59.267711 29246 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ranger-admin:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskY_TM7_/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 05:13:59.267791 29246 mini_ranger.cc:193] Using host: 127.28.143.148
I20260811 05:13:59.272135 29246 mini_ranger.cc:227] Ranger admin URL: http://127.28.143.148:38193
Aug 11, 2026 5:13:59 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 5:13:59 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 5:13:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 5:13:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 5:13:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 5:14:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 5:14:00 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 05:14:00.681084 29246 mini_ranger.cc:160] Time spent starting Ranger: real 10.583s user 0.003s sys 0.010s
Aug 11, 2026 5:14:25 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 5:14:25 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 11, 2026 5:14:25 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 5:14:25 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 5:14:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 5:14:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 5:14:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 5:14:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 5:14:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 5:14:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 5:14:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 5:14:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 5:14:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 5:14:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 5:14:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 5:14:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 5:14:25 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 5:14:26 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 05:14:27.293612 29246 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.28.143.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.28.143.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.28.143.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/krb5kdc/rangerkms_127.28.143.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.28.143.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/krb5kdc/rangerkms_127.28.143.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.28.143.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.28.143.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.28.143.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/krb5kdc/HTTP_127.28.143.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.28.143.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/krb5kdc/HTTP_127.28.143.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 05:14:27 dist-test-slave-f7th krb5kdc[16590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425267, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 05:14:27.415686 29246 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 05:14:27.430846 29246 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 05:14:28.398337 29246 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 05:14:28.398427 29246 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ranger-kms
I20260811 05:14:28.398442 29246 mini_ranger_kms.cc:227] Using postgres at 127.28.143.148:40095
2026-08-11 05:14:28,803 [I] DB FLAVOR :POSTGRES
2026-08-11 05:14:28,804 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 05:14:28,804 [I] Checking connection
2026-08-11 05:14:28,804 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:40095/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 05:14:29,121 [I] connection success
2026-08-11 05:14:29,121 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 05:14:29,121 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 05:14:29,122 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:40095/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 05:14:29,412 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 05:14:29,413 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 05:14:29,413 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 05:14:29,413 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:40095/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 05:14:29,711 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 05:14:30.323122 29246 mini_ranger_kms.cc:310] Created kms service
I20260811 05:14:30.572199 29246 mini_ranger_kms.cc:326] Created kudu user
I20260811 05:14:30.622628 29246 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 05:14:30 dist-test-slave-f7th krb5kdc[16590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425267, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.28.143.148@KRBTEST.COM
I20260811 05:14:31.113982 29246 mini_ranger_kms.cc:384] Added ranger policy
I20260811 05:14:31.114313 29246 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ranger-kms:/tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 05:14:31.114351 29246 mini_ranger_kms.cc:240] Using host: 127.28.143.148
I20260811 05:14:31.119931 29246 mini_ranger_kms.cc:276] Ranger KMS PID: 17265
I20260811 05:14:31.120092 29246 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.28.143.148:40923
Aug 11, 2026 5:14:32 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 5:14:32 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 5:14:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 5:14:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 5:14:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 5:14:32 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 05:14:33 dist-test-slave-f7th krb5kdc[16590](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786425273, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.28.143.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:14:33.750710 29246 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.335s user 0.010s sys 0.372s
I20260811 05:14:33.750834 29246 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 05:14:33.750921 29246 mini_ranger_kms.cc:401] 127.28.143.148:40923/kms/v1/keys
Aug 11 05:14:34 dist-test-slave-f7th krb5kdc[16590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425273, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.28.143.148@KRBTEST.COM for HTTP/127.28.143.148@KRBTEST.COM
Aug 11 05:14:34 dist-test-slave-f7th krb5kdc[16590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425273, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.28.143.148@KRBTEST.COM for HTTP/127.28.143.148@KRBTEST.COM
Aug 11 05:14:37 dist-test-slave-f7th krb5kdc[16590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.28.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
I20260811 05:14:37.429383 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:41749
--webserver_interface=127.28.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:33803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:41749
--ranger_config_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.143.148:40923/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:14:37.550505 17344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:14:37.550742 17344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:14:37.550766 17344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:14:37.552029 17344 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:14:37.552079 17344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:14:37.552093 17344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:14:37.552103 17344 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:14:37.552116 17344 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:14:37.554164 17344 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:33803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/master-0/wal
--ranger_kms_url=127.28.143.148:40923/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:41749
--ranger_config_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:41749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:14:37.554450 17344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:14:37.554728 17344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:14:37.561954 17357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:14:37.562013 17353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:14:37.562160 17344 server_base.cc:1034] running on GCE node
W20260811 05:14:37.562006 17355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:14:37.562512 17344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:14:37.562844 17344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:14:37.564018 17344 hybrid_clock.cc:648] HybridClock initialized: now 1786425277563998 us; error 37 us; skew 500 ppm
Aug 11 05:14:37 dist-test-slave-f7th krb5kdc[16590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425277, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:14:37.567787 17344 init.cc:377] Logged in from keytab as kudu/127.28.143.190@KRBTEST.COM (short username kudu)
I20260811 05:14:37.568573 17344 webserver.cc:460] Webserver started at http://127.28.143.190:36327/ using document root <none> and password file <none>
I20260811 05:14:37.568861 17344 fs_manager.cc:359] Metadata directory not provided
I20260811 05:14:37.568909 17344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:14:37.569199 17344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 05:14:37 dist-test-slave-f7th krb5kdc[16590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425277, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for HTTP/127.28.143.148@KRBTEST.COM
I20260811 05:14:37.634507 17344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "021e199b03124f2991fc517df3d3cb7d"
format_stamp: "Formatted at 2026-08-11 05:14:37 on dist-test-slave-f7th"
server_key: "425c961bdf0d5e2ffff3ecd25562c442"
server_key_iv: "ee642e087cf8c3da48ae85d08ed47635"
server_key_version: "kuduclusterkey@0"
I20260811 05:14:37.635025 17344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "021e199b03124f2991fc517df3d3cb7d"
format_stamp: "Formatted at 2026-08-11 05:14:37 on dist-test-slave-f7th"
server_key: "425c961bdf0d5e2ffff3ecd25562c442"
server_key_iv: "ee642e087cf8c3da48ae85d08ed47635"
server_key_version: "kuduclusterkey@0"
I20260811 05:14:37.636929 17344 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260811 05:14:37.654419 17369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:14:37.654678 17344 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 05:14:37.654819 17344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "021e199b03124f2991fc517df3d3cb7d"
format_stamp: "Formatted at 2026-08-11 05:14:37 on dist-test-slave-f7th"
server_key: "425c961bdf0d5e2ffff3ecd25562c442"
server_key_iv: "ee642e087cf8c3da48ae85d08ed47635"
server_key_version: "kuduclusterkey@0"
I20260811 05:14:37.654911 17344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:14:37.674134 17344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:14:37.676403 17344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:14:37.676600 17344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:14:37.689919 17344 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:41749
I20260811 05:14:37.690092 17467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:41749 every 8 connection(s)
I20260811 05:14:37.690543 17344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:14:37.691926 17468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:14:37.696589 17468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 021e199b03124f2991fc517df3d3cb7d: Bootstrap starting.
I20260811 05:14:37.698019 17468 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 021e199b03124f2991fc517df3d3cb7d: Neither blocks nor log segments found. Creating new log.
I20260811 05:14:37.698969 17468 log.cc:824] T 00000000000000000000000000000000 P 021e199b03124f2991fc517df3d3cb7d: Log is configured to *not* fsync() on all Append() calls
I20260811 05:14:37.699324 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 17344
I20260811 05:14:37.699446 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:14:37.699726 29246 external_mini_cluster.cc:1428] Setting key 6876bc31f5277405d5d9c6f87f48ee68
I20260811 05:14:37.700222 17468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 021e199b03124f2991fc517df3d3cb7d: No bootstrap required, opened a new log
I20260811 05:14:37.705529 17468 raft_consensus.cc:348] T 00000000000000000000000000000000 P 021e199b03124f2991fc517df3d3cb7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "021e199b03124f2991fc517df3d3cb7d" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 41749 } }
I20260811 05:14:37.705790 17468 raft_consensus.cc:374] T 00000000000000000000000000000000 P 021e199b03124f2991fc517df3d3cb7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:14:37.705825 17468 raft_consensus.cc:729] T 00000000000000000000000000000000 P 021e199b03124f2991fc517df3d3cb7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 021e199b03124f2991fc517df3d3cb7d, State: Initialized, Role: FOLLOWER
I20260811 05:14:37.706002 17468 consensus_queue.cc:260] T 00000000000000000000000000000000 P 021e199b03124f2991fc517df3d3cb7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "021e199b03124f2991fc517df3d3cb7d" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 41749 } }
I20260811 05:14:37.706115 17468 raft_consensus.cc:388] T 00000000000000000000000000000000 P 021e199b03124f2991fc517df3d3cb7d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:14:37.706163 17468 raft_consensus.cc:482] T 00000000000000000000000000000000 P 021e199b03124f2991fc517df3d3cb7d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:14:37.706307 17468 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 021e199b03124f2991fc517df3d3cb7d [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:14:37.707337 17468 raft_consensus.cc:504] T 00000000000000000000000000000000 P 021e199b03124f2991fc517df3d3cb7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "021e199b03124f2991fc517df3d3cb7d" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 41749 } }
I20260811 05:14:37.707494 17468 leader_election.cc:302] T 00000000000000000000000000000000 P 021e199b03124f2991fc517df3d3cb7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 021e199b03124f2991fc517df3d3cb7d; no voters:
I20260811 05:14:37.707751 17468 leader_election.cc:288] T 00000000000000000000000000000000 P 021e199b03124f2991fc517df3d3cb7d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:14:37.707824 17476 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 021e199b03124f2991fc517df3d3cb7d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:14:37.708149 17468 sys_catalog.cc:564] T 00000000000000000000000000000000 P 021e199b03124f2991fc517df3d3cb7d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:14:37.708251 17476 raft_consensus.cc:686] T 00000000000000000000000000000000 P 021e199b03124f2991fc517df3d3cb7d [term 1 LEADER]: Becoming Leader. State: Replica: 021e199b03124f2991fc517df3d3cb7d, State: Running, Role: LEADER
I20260811 05:14:37.708456 17476 consensus_queue.cc:237] T 00000000000000000000000000000000 P 021e199b03124f2991fc517df3d3cb7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "021e199b03124f2991fc517df3d3cb7d" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 41749 } }
I20260811 05:14:37.710129 17481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 021e199b03124f2991fc517df3d3cb7d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 021e199b03124f2991fc517df3d3cb7d. Latest consensus state: current_term: 1 leader_uuid: "021e199b03124f2991fc517df3d3cb7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "021e199b03124f2991fc517df3d3cb7d" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 41749 } } }
I20260811 05:14:37.710232 17481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 021e199b03124f2991fc517df3d3cb7d [sys.catalog]: This master's current role is: LEADER
I20260811 05:14:37.710511 17478 sys_catalog.cc:455] T 00000000000000000000000000000000 P 021e199b03124f2991fc517df3d3cb7d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "021e199b03124f2991fc517df3d3cb7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "021e199b03124f2991fc517df3d3cb7d" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 41749 } } }
I20260811 05:14:37.710585 17478 sys_catalog.cc:458] T 00000000000000000000000000000000 P 021e199b03124f2991fc517df3d3cb7d [sys.catalog]: This master's current role is: LEADER
I20260811 05:14:37.710754 17468 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 05:14:37.710992 17497 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 11 05:14:37 dist-test-slave-f7th krb5kdc[16590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:14:37.716804 17497 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:14:37.721956 17474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:37.700461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42038 (local address 127.28.143.190:41749)
0811 05:14:37.700930 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:37.700940 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:37.700994 (+ 54us) server_negotiation.cc:407] Connection header received
0811 05:14:37.705552 (+ 4558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:37.705568 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:37.705657 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:37.705869 (+ 212us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:37.706765 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:37.708398 (+ 1633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:37.709565 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:37.710231 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:37.710352 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:37.714931 (+ 4579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:14:37.715051 (+ 120us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:14:37.715052 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:14:37.715177 (+ 125us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:14:37.719522 (+ 4345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:14:37.720022 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:14:37.720026 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:14:37.720026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:14:37.720185 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:14:37.720416 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:14:37.720417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:14:37.720417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:14:37.720857 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:14:37.720999 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 05:14:37.721492 (+ 493us) server_negotiation.cc:299] Negotiation successful
0811 05:14:37.721651 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":313,"threads_started":1}
I20260811 05:14:37.728945 17497 catalog_manager.cc:1347] Generated new cluster ID: ead6bc69845243f4ae1953c50c1c0821
I20260811 05:14:37.729032 17497 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:14:37.737653 17497 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:14:37.738569 17497 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:14:37.745270 17497 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 021e199b03124f2991fc517df3d3cb7d: Generated new TSK 0
I20260811 05:14:37.745657 17497 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.143.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab.
Entry for principal kudu/127.28.143.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab.
I20260811 05:14:38.617846 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.143.148:40923/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.129:0
--local_ip_for_outbound_sockets=127.28.143.129
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:41749
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:33803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:14:38.730509 17538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:14:38.730796 17538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:14:38.730839 17538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:14:38.732264 17538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:14:38.732331 17538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:14:38.732370 17538 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.129
I20260811 05:14:38.734282 17538 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:33803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.28.143.148:40923/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:41749
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17538
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.129
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:14:38.734582 17538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:14:38.734828 17538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:14:38.741729 17554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:14:38.741729 17550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:14:38.741896 17549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:14:38.742101 17538 server_base.cc:1034] running on GCE node
I20260811 05:14:38.742496 17538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:14:38.742837 17538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:14:38.744021 17538 hybrid_clock.cc:648] HybridClock initialized: now 1786425278743998 us; error 41 us; skew 500 ppm
Aug 11 05:14:38 dist-test-slave-f7th krb5kdc[16590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425278, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:14:38.747850 17538 init.cc:377] Logged in from keytab as kudu/127.28.143.129@KRBTEST.COM (short username kudu)
I20260811 05:14:38.748831 17538 webserver.cc:460] Webserver started at http://127.28.143.129:32835/ using document root <none> and password file <none>
I20260811 05:14:38.749125 17538 fs_manager.cc:359] Metadata directory not provided
I20260811 05:14:38.749199 17538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:14:38.749361 17538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 05:14:38 dist-test-slave-f7th krb5kdc[16590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425278, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for HTTP/127.28.143.148@KRBTEST.COM
I20260811 05:14:38.765858 17538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-0/data/instance:
uuid: "5aa3fc2b2d4e4fbba094124ecbd77c1b"
format_stamp: "Formatted at 2026-08-11 05:14:38 on dist-test-slave-f7th"
server_key: "f1cd4bdb1b0516089c4e40fbac035560"
server_key_iv: "c29820f37384c4411903c80929d13416"
server_key_version: "kuduclusterkey@0"
I20260811 05:14:38.766538 17538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance:
uuid: "5aa3fc2b2d4e4fbba094124ecbd77c1b"
format_stamp: "Formatted at 2026-08-11 05:14:38 on dist-test-slave-f7th"
server_key: "f1cd4bdb1b0516089c4e40fbac035560"
server_key_iv: "c29820f37384c4411903c80929d13416"
server_key_version: "kuduclusterkey@0"
I20260811 05:14:38.769021 17538 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260811 05:14:38.784279 17570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:14:38.784557 17538 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260811 05:14:38.784734 17538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-0/wal
uuid: "5aa3fc2b2d4e4fbba094124ecbd77c1b"
format_stamp: "Formatted at 2026-08-11 05:14:38 on dist-test-slave-f7th"
server_key: "f1cd4bdb1b0516089c4e40fbac035560"
server_key_iv: "c29820f37384c4411903c80929d13416"
server_key_version: "kuduclusterkey@0"
I20260811 05:14:38.784853 17538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:14:38.803097 17538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:14:38.806896 17538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:14:38.807081 17538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:14:38.807430 17538 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:14:38.808259 17538 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:14:38.808332 17538 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:14:38.808367 17538 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:14:38.808385 17538 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:14:38.833874 17538 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.129:41741
I20260811 05:14:38.834055 17789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.129:41741 every 8 connection(s)
I20260811 05:14:38.834412 17538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
I20260811 05:14:38.838696 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 17538
I20260811 05:14:38.838826 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance
I20260811 05:14:38.839042 29246 external_mini_cluster.cc:1428] Setting key dbe761f1312f3c22b6646ad186297f4a
Aug 11 05:14:38 dist-test-slave-f7th krb5kdc[16590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425278, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:14:38.851780 17795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:38.835613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:38109 (local address 127.28.143.190:41749)
0811 05:14:38.835749 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:38.835752 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:38.836111 (+ 359us) server_negotiation.cc:407] Connection header received
0811 05:14:38.837120 (+ 1009us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:38.837122 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:38.837142 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:38.837212 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:38.838520 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:38.839387 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:38.840987 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:38.841296 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:38.841361 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:38.847614 (+ 6253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:14:38.847617 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:14:38.847617 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:14:38.847628 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:14:38.849702 (+ 2074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:14:38.850377 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:14:38.850379 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:14:38.850379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:14:38.850459 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:14:38.850813 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:14:38.850815 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:14:38.850815 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:14:38.850999 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:14:38.851057 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 05:14:38.851527 (+ 470us) server_negotiation.cc:299] Negotiation successful
0811 05:14:38.851635 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":67,"threads_started":1}
I20260811 05:14:38.852875 17796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:38.835547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:41749 (local address 127.28.143.129:38109)
0811 05:14:38.835897 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:38.835946 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:38.836968 (+ 1022us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:38.837306 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:38.837310 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:38.837828 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:38.838273 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:38.838281 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:38.839530 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:38.839532 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:38.840854 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:38.840859 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:38.844676 (+ 3817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:38.844678 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:38.844942 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:38.845606 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:38.845619 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:38.847451 (+ 1832us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:14:38.849860 (+ 2409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:14:38.849876 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:14:38.849880 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:14:38.850261 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:14:38.850540 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:14:38.850541 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:14:38.850542 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:14:38.850725 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:14:38.851100 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:14:38.851111 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:14:38.851331 (+ 220us) client_negotiation.cc:769] Sending connection context
0811 05:14:38.852235 (+ 904us) client_negotiation.cc:240] Negotiation successful
0811 05:14:38.852469 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":219,"threads_started":1}
I20260811 05:14:38.853818 17793 heartbeater.cc:347] Connected to a master server at 127.28.143.190:41749
I20260811 05:14:38.853956 17793 heartbeater.cc:464] Registering TS with master...
I20260811 05:14:38.854202 17793 heartbeater.cc:511] Master 127.28.143.190:41749 requested a full tablet report, sending...
I20260811 05:14:38.854961 17400 ts_manager.cc:194] Registered new tserver with Master: 5aa3fc2b2d4e4fbba094124ecbd77c1b (127.28.143.129:41741)
WARNING: no policy specified for kudu/127.28.143.130@KRBTEST.COM; defaulting to no policy
I20260811 05:14:38.855819 17400 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.129@KRBTEST.COM'} at 127.28.143.129:38109
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab.
Entry for principal kudu/127.28.143.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab.
I20260811 05:14:38.880708 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.143.148:40923/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.130:0
--local_ip_for_outbound_sockets=127.28.143.130
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:41749
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:33803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:14:39.033913 17800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:14:39.034328 17800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:14:39.034408 17800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
Aug 11 05:14:39 dist-test-slave-f7th krb5kdc[16590](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786425279, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 05:14:39.036473 17800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:14:39.036600 17800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:14:39.036729 17800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.130
I20260811 05:14:39.039361 17800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:33803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.28.143.148:40923/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:41749
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.130
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:14:39.039889 17800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:14:39.040242 17800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:14:39.047914 17819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:14:39.048143 17822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:14:39.048391 17816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:14:39.049332 17800 server_base.cc:1034] running on GCE node
I20260811 05:14:39.049708 17800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:14:39.050053 17800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:14:39.051235 17800 hybrid_clock.cc:648] HybridClock initialized: now 1786425279051214 us; error 37 us; skew 500 ppm
Aug 11 05:14:39 dist-test-slave-f7th krb5kdc[16590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425279, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:14:39.054903 17800 init.cc:377] Logged in from keytab as kudu/127.28.143.130@KRBTEST.COM (short username kudu)
I20260811 05:14:39.055639 17800 webserver.cc:460] Webserver started at http://127.28.143.130:38351/ using document root <none> and password file <none>
I20260811 05:14:39.056016 17800 fs_manager.cc:359] Metadata directory not provided
I20260811 05:14:39.056143 17800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:14:39.056339 17800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 05:14:39 dist-test-slave-f7th krb5kdc[16590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425279, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for HTTP/127.28.143.148@KRBTEST.COM
I20260811 05:14:39.093385 17800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-1/data/instance:
uuid: "4259d3d7a07445fb91bcf9cff27a444c"
format_stamp: "Formatted at 2026-08-11 05:14:39 on dist-test-slave-f7th"
server_key: "a786e98e915bba55e34d74e918246575"
server_key_iv: "8ded02a42cccafb1d023b628bdac5e86"
server_key_version: "kuduclusterkey@0"
I20260811 05:14:39.094772 17800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance:
uuid: "4259d3d7a07445fb91bcf9cff27a444c"
format_stamp: "Formatted at 2026-08-11 05:14:39 on dist-test-slave-f7th"
server_key: "a786e98e915bba55e34d74e918246575"
server_key_iv: "8ded02a42cccafb1d023b628bdac5e86"
server_key_version: "kuduclusterkey@0"
I20260811 05:14:39.104887 17800 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.001s sys 0.001s
I20260811 05:14:39.121019 17831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:14:39.121220 17800 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260811 05:14:39.121333 17800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-1/wal
uuid: "4259d3d7a07445fb91bcf9cff27a444c"
format_stamp: "Formatted at 2026-08-11 05:14:39 on dist-test-slave-f7th"
server_key: "a786e98e915bba55e34d74e918246575"
server_key_iv: "8ded02a42cccafb1d023b628bdac5e86"
server_key_version: "kuduclusterkey@0"
I20260811 05:14:39.121434 17800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:14:39.142647 17800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:14:39.145905 17800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:14:39.146099 17800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:14:39.146517 17800 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:14:39.147074 17800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:14:39.147130 17800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:14:39.147163 17800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:14:39.147223 17800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:14:39.170121 17800 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.130:43415
I20260811 05:14:39.170161 18026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.130:43415 every 8 connection(s)
I20260811 05:14:39.170989 17800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
I20260811 05:14:39.179863 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 17800
I20260811 05:14:39.179996 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance
I20260811 05:14:39.180193 29246 external_mini_cluster.cc:1428] Setting key 8dacc3a4bb71907fc9675ec3320e4f5f
Aug 11 05:14:39 dist-test-slave-f7th krb5kdc[16590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425279, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:14:39.193080 17795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:39.172766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:49839 (local address 127.28.143.190:41749)
0811 05:14:39.172814 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:39.172816 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:39.172826 (+ 10us) server_negotiation.cc:407] Connection header received
0811 05:14:39.174619 (+ 1793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:39.174621 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:39.174643 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:39.174722 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:39.175683 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:39.176492 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:39.178301 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:39.178703 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:39.178753 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:39.188943 (+ 10190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:14:39.188945 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:14:39.188946 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:14:39.188957 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:14:39.190874 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:14:39.191580 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:14:39.191582 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:14:39.191582 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:14:39.191657 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:14:39.192050 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:14:39.192052 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:14:39.192053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:14:39.192259 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:14:39.192320 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 05:14:39.192825 (+ 505us) server_negotiation.cc:299] Negotiation successful
0811 05:14:39.192947 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 05:14:39.194144 18031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:39.172245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:41749 (local address 127.28.143.130:49839)
0811 05:14:39.172582 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:39.172651 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:39.173532 (+ 881us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:39.174838 (+ 1306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:39.174843 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:39.175307 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:39.175567 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:39.175573 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:39.176649 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:39.176651 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:39.178167 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:39.178171 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:39.178868 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:39.178869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:39.179106 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:39.185730 (+ 6624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:39.185750 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:39.188758 (+ 3008us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:14:39.191024 (+ 2266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:14:39.191039 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:14:39.191043 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:14:39.191453 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:14:39.191778 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:14:39.191779 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:14:39.191779 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:14:39.191962 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:14:39.192364 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:14:39.192374 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:14:39.192638 (+ 264us) client_negotiation.cc:769] Sending connection context
0811 05:14:39.193527 (+ 889us) client_negotiation.cc:240] Negotiation successful
0811 05:14:39.193765 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":248,"threads_started":1}
I20260811 05:14:39.195106 18027 heartbeater.cc:347] Connected to a master server at 127.28.143.190:41749
I20260811 05:14:39.195231 18027 heartbeater.cc:464] Registering TS with master...
I20260811 05:14:39.195441 18027 heartbeater.cc:511] Master 127.28.143.190:41749 requested a full tablet report, sending...
I20260811 05:14:39.195873 17400 ts_manager.cc:194] Registered new tserver with Master: 4259d3d7a07445fb91bcf9cff27a444c (127.28.143.130:43415)
I20260811 05:14:39.196394 17400 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.130@KRBTEST.COM'} at 127.28.143.130:49839
WARNING: no policy specified for kudu/127.28.143.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab.
Entry for principal kudu/127.28.143.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab.
I20260811 05:14:39.231719 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.143.148:40923/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.131:0
--local_ip_for_outbound_sockets=127.28.143.131
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:41749
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:33803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:14:39.357254 18034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:14:39.357666 18034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:14:39.357728 18034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:14:39.359995 18034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:14:39.360082 18034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:14:39.360131 18034 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.131
I20260811 05:14:39.363109 18034 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:33803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.28.143.148:40923/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:41749
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18034
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.131
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:14:39.363448 18034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:14:39.363772 18034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:14:39.382807 18043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:14:39.382800 18042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:14:39.384323 18047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:14:39.390699 18034 server_base.cc:1034] running on GCE node
I20260811 05:14:39.391062 18034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:14:39.391439 18034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:14:39.392791 18034 hybrid_clock.cc:648] HybridClock initialized: now 1786425279392743 us; error 56 us; skew 500 ppm
Aug 11 05:14:39 dist-test-slave-f7th krb5kdc[16590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425279, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:14:39.400458 18034 init.cc:377] Logged in from keytab as kudu/127.28.143.131@KRBTEST.COM (short username kudu)
I20260811 05:14:39.401296 18034 webserver.cc:460] Webserver started at http://127.28.143.131:36107/ using document root <none> and password file <none>
I20260811 05:14:39.401712 18034 fs_manager.cc:359] Metadata directory not provided
I20260811 05:14:39.401870 18034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:14:39.402096 18034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 05:14:39 dist-test-slave-f7th krb5kdc[16590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425279, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for HTTP/127.28.143.148@KRBTEST.COM
I20260811 05:14:39.424556 18034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-2/data/instance:
uuid: "08b740d5997d4132a29fd1a14ca86e80"
format_stamp: "Formatted at 2026-08-11 05:14:39 on dist-test-slave-f7th"
server_key: "8515981d94636cedb171981009089fe3"
server_key_iv: "444cd85e06bea9f30a6d0afbe5655648"
server_key_version: "kuduclusterkey@0"
I20260811 05:14:39.425127 18034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance:
uuid: "08b740d5997d4132a29fd1a14ca86e80"
format_stamp: "Formatted at 2026-08-11 05:14:39 on dist-test-slave-f7th"
server_key: "8515981d94636cedb171981009089fe3"
server_key_iv: "444cd85e06bea9f30a6d0afbe5655648"
server_key_version: "kuduclusterkey@0"
I20260811 05:14:39.427400 18034 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:14:39.444000 18058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:14:39.444242 18034 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:14:39.444331 18034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-2/wal
uuid: "08b740d5997d4132a29fd1a14ca86e80"
format_stamp: "Formatted at 2026-08-11 05:14:39 on dist-test-slave-f7th"
server_key: "8515981d94636cedb171981009089fe3"
server_key_iv: "444cd85e06bea9f30a6d0afbe5655648"
server_key_version: "kuduclusterkey@0"
I20260811 05:14:39.444437 18034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:14:39.458874 18034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:14:39.461771 18034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:14:39.461956 18034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:14:39.462283 18034 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:14:39.462713 18034 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:14:39.462764 18034 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:14:39.462797 18034 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:14:39.462821 18034 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:14:39.507861 18034 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.131:34219
I20260811 05:14:39.508415 18034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
I20260811 05:14:39.509750 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 18034
I20260811 05:14:39.509855 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance
I20260811 05:14:39.510107 29246 external_mini_cluster.cc:1428] Setting key af3fb237be4946c79b5bb23a2322b5c9
I20260811 05:14:39.511904 18263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.131:34219 every 8 connection(s)
Aug 11 05:14:39 dist-test-slave-f7th krb5kdc[16590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425279, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:14:39.525081 17795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:39.512682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:43949 (local address 127.28.143.190:41749)
0811 05:14:39.512743 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:39.512745 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:39.512757 (+ 12us) server_negotiation.cc:407] Connection header received
0811 05:14:39.513506 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:39.513507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:39.513521 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:39.513562 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:39.514344 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:39.514858 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:39.516502 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:39.516774 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:39.516837 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:39.519932 (+ 3095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:14:39.519935 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:14:39.519935 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:14:39.519947 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:14:39.522197 (+ 2250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:14:39.522908 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:14:39.522910 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:14:39.522911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:14:39.522985 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:14:39.523327 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:14:39.523328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:14:39.523328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:14:39.523498 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:14:39.524309 (+ 811us) server_negotiation.cc:1035] Waiting for connection context
0811 05:14:39.524839 (+ 530us) server_negotiation.cc:299] Negotiation successful
0811 05:14:39.524975 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 05:14:39.526042 18267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:39.511973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:41749 (local address 127.28.143.131:43949)
0811 05:14:39.512399 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:39.512465 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:39.513323 (+ 858us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:39.513662 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:39.513667 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:39.513938 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:39.514250 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:39.514256 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:39.515027 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:39.515029 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:39.516392 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:39.516397 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:39.516930 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:39.516932 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:39.517101 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:39.517820 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:39.517833 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:39.519777 (+ 1944us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:14:39.522344 (+ 2567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:14:39.522361 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:14:39.522365 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:14:39.522757 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:14:39.523067 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:14:39.523068 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:14:39.523068 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:14:39.523242 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:14:39.524366 (+ 1124us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:14:39.524377 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:14:39.524596 (+ 219us) client_negotiation.cc:769] Sending connection context
0811 05:14:39.525473 (+ 877us) client_negotiation.cc:240] Negotiation successful
0811 05:14:39.525721 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":268,"threads_started":1}
I20260811 05:14:39.527133 18264 heartbeater.cc:347] Connected to a master server at 127.28.143.190:41749
I20260811 05:14:39.527310 18264 heartbeater.cc:464] Registering TS with master...
I20260811 05:14:39.527592 18264 heartbeater.cc:511] Master 127.28.143.190:41749 requested a full tablet report, sending...
I20260811 05:14:39.528127 17400 ts_manager.cc:194] Registered new tserver with Master: 08b740d5997d4132a29fd1a14ca86e80 (127.28.143.131:34219)
I20260811 05:14:39.528807 17400 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.131@KRBTEST.COM'} at 127.28.143.131:43949
I20260811 05:14:39.535439 29246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:14:39.546656 17795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:39.537708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42052 (local address 127.28.143.190:41749)
0811 05:14:39.537765 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:39.537768 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:39.537986 (+ 218us) server_negotiation.cc:407] Connection header received
0811 05:14:39.538074 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:39.538076 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:39.538091 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:39.538152 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:39.538991 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:39.539762 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:39.540997 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:39.541303 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:39.541350 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:39.543117 (+ 1767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:14:39.543119 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:14:39.543120 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:14:39.543131 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:14:39.545167 (+ 2036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:14:39.545668 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:14:39.545670 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:14:39.545670 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:14:39.545749 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:14:39.545987 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:14:39.545989 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:14:39.545989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:14:39.546160 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:14:39.546275 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 05:14:39.546447 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 05:14:39.546548 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 05:14:39.549409 17400 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42052:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:14:39.552088 17400 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:14:39.563676 18283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:39.555654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:41741 (local address 127.0.0.1:38188)
0811 05:14:39.557153 (+ 1499us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:39.557206 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:39.557283 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:39.557633 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:39.557637 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:39.557639 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:14:39.557725 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:39.557728 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:39.559234 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:39.559236 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:39.560893 (+ 1657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:39.560898 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:39.563303 (+ 2405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:39.563305 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:39.563426 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:39.563447 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 05:14:39.563520 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 05:14:39.563584 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1455,"thread_start_us":1259,"threads_started":1}
I20260811 05:14:39.564180 18282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:39.555966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38188 (local address 127.28.143.129:41741)
0811 05:14:39.557436 (+ 1470us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:39.557440 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:39.557451 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:14:39.557487 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:39.557489 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:39.557506 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:39.557604 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:14:39.557880 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:39.559135 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:39.561003 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:39.561769 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:39.561842 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:39.561916 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 05:14:39.564050 (+ 2134us) server_negotiation.cc:299] Negotiation successful
0811 05:14:39.564111 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1432,"thread_start_us":86,"threads_started":1}
I20260811 05:14:39.565287 17659 tablet_service.cc:1467] Processing CreateTablet for tablet 0ee49582dba142f3bf8ed4023ce00c4f (DEFAULT_TABLE table=test-table [id=b3b7f561d7924805aa3c07d7268ce15b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:14:39.565785 17659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ee49582dba142f3bf8ed4023ce00c4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:14:39.566705 18288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:39.555660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.131:34219 (local address 127.0.0.1:38902)
0811 05:14:39.557251 (+ 1591us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:39.557263 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:39.557324 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:39.560488 (+ 3164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:39.560490 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:39.560493 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:14:39.560554 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:39.560556 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:39.563473 (+ 2917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:39.563475 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:39.565096 (+ 1621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:39.565100 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:39.566415 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:39.566417 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:39.566491 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:39.566496 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 05:14:39.566565 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 05:14:39.566619 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1561,"thread_start_us":1088,"threads_started":1}
I20260811 05:14:39.567648 18285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:39.556151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38902 (local address 127.28.143.131:34219)
0811 05:14:39.557759 (+ 1608us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:39.557763 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:39.557772 (+ 9us) server_negotiation.cc:407] Connection header received
0811 05:14:39.557814 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:39.557816 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:39.559521 (+ 1705us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:39.559608 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:14:39.561974 (+ 2366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:39.563178 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:39.565275 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:39.566144 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:39.566297 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:39.566370 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 05:14:39.567490 (+ 1120us) server_negotiation.cc:299] Negotiation successful
0811 05:14:39.567560 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1564,"thread_start_us":107,"threads_started":1}
I20260811 05:14:39.568478 18121 tablet_service.cc:1467] Processing CreateTablet for tablet 0ee49582dba142f3bf8ed4023ce00c4f (DEFAULT_TABLE table=test-table [id=b3b7f561d7924805aa3c07d7268ce15b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:14:39.568897 18121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ee49582dba142f3bf8ed4023ce00c4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:14:39.569481 18286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:39.556274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50242 (local address 127.28.143.130:43415)
0811 05:14:39.557062 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:39.557068 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:39.564861 (+ 7793us) server_negotiation.cc:407] Connection header received
0811 05:14:39.565006 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:39.565009 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:39.565029 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:39.565087 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:14:39.565347 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:39.566590 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:39.568011 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:39.568909 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:39.569005 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:39.569070 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 05:14:39.569305 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 05:14:39.569351 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":736,"thread_start_us":213,"threads_started":1}
I20260811 05:14:39.569737 18283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:39.555798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:43415 (local address 127.0.0.1:50242)
0811 05:14:39.564778 (+ 8980us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:39.564790 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:39.564894 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:39.565229 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:39.565230 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:39.565232 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:14:39.565269 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:39.565271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:39.566754 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:39.566755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:39.567800 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:39.567804 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:39.569143 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:39.569145 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:39.569211 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:39.569215 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:14:39.569609 (+ 394us) client_negotiation.cc:240] Negotiation successful
0811 05:14:39.569669 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8965,"thread_start_us":264,"threads_started":1}
I20260811 05:14:39.570187 18291 tablet_bootstrap.cc:492] T 0ee49582dba142f3bf8ed4023ce00c4f P 5aa3fc2b2d4e4fbba094124ecbd77c1b: Bootstrap starting.
I20260811 05:14:39.570525 17913 tablet_service.cc:1467] Processing CreateTablet for tablet 0ee49582dba142f3bf8ed4023ce00c4f (DEFAULT_TABLE table=test-table [id=b3b7f561d7924805aa3c07d7268ce15b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:14:39.571033 17913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ee49582dba142f3bf8ed4023ce00c4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:14:39.571237 18291 tablet_bootstrap.cc:654] T 0ee49582dba142f3bf8ed4023ce00c4f P 5aa3fc2b2d4e4fbba094124ecbd77c1b: Neither blocks nor log segments found. Creating new log.
I20260811 05:14:39.571738 18291 log.cc:824] T 0ee49582dba142f3bf8ed4023ce00c4f P 5aa3fc2b2d4e4fbba094124ecbd77c1b: Log is configured to *not* fsync() on all Append() calls
I20260811 05:14:39.575508 18295 tablet_bootstrap.cc:492] T 0ee49582dba142f3bf8ed4023ce00c4f P 4259d3d7a07445fb91bcf9cff27a444c: Bootstrap starting.
I20260811 05:14:39.575809 18294 tablet_bootstrap.cc:492] T 0ee49582dba142f3bf8ed4023ce00c4f P 08b740d5997d4132a29fd1a14ca86e80: Bootstrap starting.
I20260811 05:14:39.577241 18294 tablet_bootstrap.cc:654] T 0ee49582dba142f3bf8ed4023ce00c4f P 08b740d5997d4132a29fd1a14ca86e80: Neither blocks nor log segments found. Creating new log.
I20260811 05:14:39.577847 18294 log.cc:824] T 0ee49582dba142f3bf8ed4023ce00c4f P 08b740d5997d4132a29fd1a14ca86e80: Log is configured to *not* fsync() on all Append() calls
I20260811 05:14:39.578030 18295 tablet_bootstrap.cc:654] T 0ee49582dba142f3bf8ed4023ce00c4f P 4259d3d7a07445fb91bcf9cff27a444c: Neither blocks nor log segments found. Creating new log.
I20260811 05:14:39.578619 18295 log.cc:824] T 0ee49582dba142f3bf8ed4023ce00c4f P 4259d3d7a07445fb91bcf9cff27a444c: Log is configured to *not* fsync() on all Append() calls
I20260811 05:14:39.578850 18294 tablet_bootstrap.cc:492] T 0ee49582dba142f3bf8ed4023ce00c4f P 08b740d5997d4132a29fd1a14ca86e80: No bootstrap required, opened a new log
I20260811 05:14:39.578940 18294 ts_tablet_manager.cc:1397] T 0ee49582dba142f3bf8ed4023ce00c4f P 08b740d5997d4132a29fd1a14ca86e80: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260811 05:14:39.581423 18291 tablet_bootstrap.cc:492] T 0ee49582dba142f3bf8ed4023ce00c4f P 5aa3fc2b2d4e4fbba094124ecbd77c1b: No bootstrap required, opened a new log
I20260811 05:14:39.581532 18291 ts_tablet_manager.cc:1397] T 0ee49582dba142f3bf8ed4023ce00c4f P 5aa3fc2b2d4e4fbba094124ecbd77c1b: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20260811 05:14:39.581847 18294 raft_consensus.cc:348] T 0ee49582dba142f3bf8ed4023ce00c4f P 08b740d5997d4132a29fd1a14ca86e80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aa3fc2b2d4e4fbba094124ecbd77c1b" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 41741 } } peers { permanent_uuid: "4259d3d7a07445fb91bcf9cff27a444c" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 43415 } } peers { permanent_uuid: "08b740d5997d4132a29fd1a14ca86e80" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 34219 } }
I20260811 05:14:39.582142 18294 raft_consensus.cc:374] T 0ee49582dba142f3bf8ed4023ce00c4f P 08b740d5997d4132a29fd1a14ca86e80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:14:39.582194 18294 raft_consensus.cc:729] T 0ee49582dba142f3bf8ed4023ce00c4f P 08b740d5997d4132a29fd1a14ca86e80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08b740d5997d4132a29fd1a14ca86e80, State: Initialized, Role: FOLLOWER
I20260811 05:14:39.582369 18294 consensus_queue.cc:260] T 0ee49582dba142f3bf8ed4023ce00c4f P 08b740d5997d4132a29fd1a14ca86e80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aa3fc2b2d4e4fbba094124ecbd77c1b" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 41741 } } peers { permanent_uuid: "4259d3d7a07445fb91bcf9cff27a444c" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 43415 } } peers { permanent_uuid: "08b740d5997d4132a29fd1a14ca86e80" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 34219 } }
I20260811 05:14:39.582852 18294 ts_tablet_manager.cc:1428] T 0ee49582dba142f3bf8ed4023ce00c4f P 08b740d5997d4132a29fd1a14ca86e80: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260811 05:14:39.582930 18264 heartbeater.cc:503] Master 127.28.143.190:41749 was elected leader, sending a full tablet report...
I20260811 05:14:39.583954 18295 tablet_bootstrap.cc:492] T 0ee49582dba142f3bf8ed4023ce00c4f P 4259d3d7a07445fb91bcf9cff27a444c: No bootstrap required, opened a new log
I20260811 05:14:39.584064 18295 ts_tablet_manager.cc:1397] T 0ee49582dba142f3bf8ed4023ce00c4f P 4259d3d7a07445fb91bcf9cff27a444c: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.003s
I20260811 05:14:39.585036 18291 raft_consensus.cc:348] T 0ee49582dba142f3bf8ed4023ce00c4f P 5aa3fc2b2d4e4fbba094124ecbd77c1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aa3fc2b2d4e4fbba094124ecbd77c1b" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 41741 } } peers { permanent_uuid: "4259d3d7a07445fb91bcf9cff27a444c" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 43415 } } peers { permanent_uuid: "08b740d5997d4132a29fd1a14ca86e80" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 34219 } }
I20260811 05:14:39.585314 18291 raft_consensus.cc:374] T 0ee49582dba142f3bf8ed4023ce00c4f P 5aa3fc2b2d4e4fbba094124ecbd77c1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:14:39.585421 18291 raft_consensus.cc:729] T 0ee49582dba142f3bf8ed4023ce00c4f P 5aa3fc2b2d4e4fbba094124ecbd77c1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5aa3fc2b2d4e4fbba094124ecbd77c1b, State: Initialized, Role: FOLLOWER
I20260811 05:14:39.585698 18291 consensus_queue.cc:260] T 0ee49582dba142f3bf8ed4023ce00c4f P 5aa3fc2b2d4e4fbba094124ecbd77c1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aa3fc2b2d4e4fbba094124ecbd77c1b" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 41741 } } peers { permanent_uuid: "4259d3d7a07445fb91bcf9cff27a444c" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 43415 } } peers { permanent_uuid: "08b740d5997d4132a29fd1a14ca86e80" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 34219 } }
I20260811 05:14:39.586226 18291 ts_tablet_manager.cc:1428] T 0ee49582dba142f3bf8ed4023ce00c4f P 5aa3fc2b2d4e4fbba094124ecbd77c1b: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260811 05:14:39.586424 17793 heartbeater.cc:503] Master 127.28.143.190:41749 was elected leader, sending a full tablet report...
I20260811 05:14:39.588855 18295 raft_consensus.cc:348] T 0ee49582dba142f3bf8ed4023ce00c4f P 4259d3d7a07445fb91bcf9cff27a444c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aa3fc2b2d4e4fbba094124ecbd77c1b" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 41741 } } peers { permanent_uuid: "4259d3d7a07445fb91bcf9cff27a444c" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 43415 } } peers { permanent_uuid: "08b740d5997d4132a29fd1a14ca86e80" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 34219 } }
I20260811 05:14:39.589226 18295 raft_consensus.cc:374] T 0ee49582dba142f3bf8ed4023ce00c4f P 4259d3d7a07445fb91bcf9cff27a444c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:14:39.589303 18295 raft_consensus.cc:729] T 0ee49582dba142f3bf8ed4023ce00c4f P 4259d3d7a07445fb91bcf9cff27a444c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4259d3d7a07445fb91bcf9cff27a444c, State: Initialized, Role: FOLLOWER
I20260811 05:14:39.589497 18295 consensus_queue.cc:260] T 0ee49582dba142f3bf8ed4023ce00c4f P 4259d3d7a07445fb91bcf9cff27a444c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aa3fc2b2d4e4fbba094124ecbd77c1b" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 41741 } } peers { permanent_uuid: "4259d3d7a07445fb91bcf9cff27a444c" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 43415 } } peers { permanent_uuid: "08b740d5997d4132a29fd1a14ca86e80" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 34219 } }
I20260811 05:14:39.590057 18295 ts_tablet_manager.cc:1428] T 0ee49582dba142f3bf8ed4023ce00c4f P 4259d3d7a07445fb91bcf9cff27a444c: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 05:14:39.590070 18027 heartbeater.cc:503] Master 127.28.143.190:41749 was elected leader, sending a full tablet report...
Aug 11 05:14:39 dist-test-slave-f7th krb5kdc[16590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425279, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for HTTP/127.28.143.148@KRBTEST.COM
I20260811 05:14:39.673154 18303 raft_consensus.cc:482] T 0ee49582dba142f3bf8ed4023ce00c4f P 5aa3fc2b2d4e4fbba094124ecbd77c1b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:14:39.673334 18303 raft_consensus.cc:504] T 0ee49582dba142f3bf8ed4023ce00c4f P 5aa3fc2b2d4e4fbba094124ecbd77c1b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aa3fc2b2d4e4fbba094124ecbd77c1b" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 41741 } } peers { permanent_uuid: "4259d3d7a07445fb91bcf9cff27a444c" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 43415 } } peers { permanent_uuid: "08b740d5997d4132a29fd1a14ca86e80" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 34219 } }
I20260811 05:14:39.673810 18303 leader_election.cc:288] T 0ee49582dba142f3bf8ed4023ce00c4f P 5aa3fc2b2d4e4fbba094124ecbd77c1b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4259d3d7a07445fb91bcf9cff27a444c (127.28.143.130:43415), 08b740d5997d4132a29fd1a14ca86e80 (127.28.143.131:34219)
I20260811 05:14:39.679035 18285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:39.674203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:48837 (local address 127.28.143.131:34219)
0811 05:14:39.674247 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:39.674249 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:39.674482 (+ 233us) server_negotiation.cc:407] Connection header received
0811 05:14:39.674591 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:39.674593 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:39.674605 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:39.674682 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:14:39.675163 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:39.675980 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:39.677968 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:39.678523 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:39.678581 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:39.678592 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 05:14:39.678898 (+ 306us) server_negotiation.cc:299] Negotiation successful
0811 05:14:39.678952 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 05:14:39.679329 18286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:39.674072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:36519 (local address 127.28.143.130:43415)
0811 05:14:39.674122 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:39.674124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:39.675469 (+ 1345us) server_negotiation.cc:407] Connection header received
0811 05:14:39.675589 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:39.675591 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:39.675602 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:39.675632 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:14:39.675983 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:39.676708 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:39.678298 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:39.678825 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:39.678877 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:39.678887 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 05:14:39.679212 (+ 325us) server_negotiation.cc:299] Negotiation successful
0811 05:14:39.679262 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 05:14:39.679766 18307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:39.673964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.131:34219 (local address 127.28.143.129:48837)
0811 05:14:39.674404 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:39.674418 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:39.674517 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:39.674694 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:39.674695 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:39.674774 (+ 79us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:14:39.675055 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:39.675059 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:39.676230 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:39.676231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:39.677841 (+ 1610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:39.677844 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:39.678656 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:39.678657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:39.678795 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:39.678801 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 05:14:39.679378 (+ 577us) client_negotiation.cc:240] Negotiation successful
0811 05:14:39.679550 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"mutex_wait_us":1,"thread_start_us":172,"threads_started":1}
I20260811 05:14:39.680419 18166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ee49582dba142f3bf8ed4023ce00c4f" candidate_uuid: "5aa3fc2b2d4e4fbba094124ecbd77c1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08b740d5997d4132a29fd1a14ca86e80" is_pre_election: true
I20260811 05:14:39.680678 18166 raft_consensus.cc:2454] T 0ee49582dba142f3bf8ed4023ce00c4f P 08b740d5997d4132a29fd1a14ca86e80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5aa3fc2b2d4e4fbba094124ecbd77c1b in term 0.
I20260811 05:14:39.681047 17571 leader_election.cc:302] T 0ee49582dba142f3bf8ed4023ce00c4f P 5aa3fc2b2d4e4fbba094124ecbd77c1b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08b740d5997d4132a29fd1a14ca86e80, 5aa3fc2b2d4e4fbba094124ecbd77c1b; no voters:
I20260811 05:14:39.679802 18306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:39.673972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:43415 (local address 127.28.143.129:36519)
0811 05:14:39.675397 (+ 1425us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:39.675409 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:39.675511 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:39.675700 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:39.675701 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:39.675703 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:14:39.675883 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:39.675886 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:39.676946 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:39.676947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:39.678182 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:39.678187 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:39.678948 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:39.678949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:39.679060 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:39.679064 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:14:39.679467 (+ 403us) client_negotiation.cc:240] Negotiation successful
0811 05:14:39.679529 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1392,"thread_start_us":232,"threads_started":1}
I20260811 05:14:39.681836 17949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ee49582dba142f3bf8ed4023ce00c4f" candidate_uuid: "5aa3fc2b2d4e4fbba094124ecbd77c1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4259d3d7a07445fb91bcf9cff27a444c" is_pre_election: true
I20260811 05:14:39.682093 17949 raft_consensus.cc:2454] T 0ee49582dba142f3bf8ed4023ce00c4f P 4259d3d7a07445fb91bcf9cff27a444c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5aa3fc2b2d4e4fbba094124ecbd77c1b in term 0.
W20260811 05:14:39.682859 18028 tablet.cc:2369] T 0ee49582dba142f3bf8ed4023ce00c4f P 4259d3d7a07445fb91bcf9cff27a444c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:14:39.683971 18303 raft_consensus.cc:2781] T 0ee49582dba142f3bf8ed4023ce00c4f P 5aa3fc2b2d4e4fbba094124ecbd77c1b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:14:39.684079 18303 raft_consensus.cc:482] T 0ee49582dba142f3bf8ed4023ce00c4f P 5aa3fc2b2d4e4fbba094124ecbd77c1b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:14:39.684105 18303 raft_consensus.cc:3037] T 0ee49582dba142f3bf8ed4023ce00c4f P 5aa3fc2b2d4e4fbba094124ecbd77c1b [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:14:39.685304 18303 raft_consensus.cc:504] T 0ee49582dba142f3bf8ed4023ce00c4f P 5aa3fc2b2d4e4fbba094124ecbd77c1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aa3fc2b2d4e4fbba094124ecbd77c1b" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 41741 } } peers { permanent_uuid: "4259d3d7a07445fb91bcf9cff27a444c" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 43415 } } peers { permanent_uuid: "08b740d5997d4132a29fd1a14ca86e80" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 34219 } }
I20260811 05:14:39.685482 18303 leader_election.cc:288] T 0ee49582dba142f3bf8ed4023ce00c4f P 5aa3fc2b2d4e4fbba094124ecbd77c1b [CANDIDATE]: Term 1 election: Requested vote from peers 4259d3d7a07445fb91bcf9cff27a444c (127.28.143.130:43415), 08b740d5997d4132a29fd1a14ca86e80 (127.28.143.131:34219)
I20260811 05:14:39.685732 17949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ee49582dba142f3bf8ed4023ce00c4f" candidate_uuid: "5aa3fc2b2d4e4fbba094124ecbd77c1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4259d3d7a07445fb91bcf9cff27a444c"
I20260811 05:14:39.685798 18166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ee49582dba142f3bf8ed4023ce00c4f" candidate_uuid: "5aa3fc2b2d4e4fbba094124ecbd77c1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08b740d5997d4132a29fd1a14ca86e80"
I20260811 05:14:39.685829 17949 raft_consensus.cc:3037] T 0ee49582dba142f3bf8ed4023ce00c4f P 4259d3d7a07445fb91bcf9cff27a444c [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:14:39.685916 18166 raft_consensus.cc:3037] T 0ee49582dba142f3bf8ed4023ce00c4f P 08b740d5997d4132a29fd1a14ca86e80 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:14:39.687075 17949 raft_consensus.cc:2454] T 0ee49582dba142f3bf8ed4023ce00c4f P 4259d3d7a07445fb91bcf9cff27a444c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5aa3fc2b2d4e4fbba094124ecbd77c1b in term 1.
I20260811 05:14:39.687156 18166 raft_consensus.cc:2454] T 0ee49582dba142f3bf8ed4023ce00c4f P 08b740d5997d4132a29fd1a14ca86e80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5aa3fc2b2d4e4fbba094124ecbd77c1b in term 1.
I20260811 05:14:39.687294 17573 leader_election.cc:302] T 0ee49582dba142f3bf8ed4023ce00c4f P 5aa3fc2b2d4e4fbba094124ecbd77c1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4259d3d7a07445fb91bcf9cff27a444c, 5aa3fc2b2d4e4fbba094124ecbd77c1b; no voters:
I20260811 05:14:39.687597 18303 raft_consensus.cc:2781] T 0ee49582dba142f3bf8ed4023ce00c4f P 5aa3fc2b2d4e4fbba094124ecbd77c1b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:14:39.687949 18303 raft_consensus.cc:686] T 0ee49582dba142f3bf8ed4023ce00c4f P 5aa3fc2b2d4e4fbba094124ecbd77c1b [term 1 LEADER]: Becoming Leader. State: Replica: 5aa3fc2b2d4e4fbba094124ecbd77c1b, State: Running, Role: LEADER
I20260811 05:14:39.688086 18303 consensus_queue.cc:237] T 0ee49582dba142f3bf8ed4023ce00c4f P 5aa3fc2b2d4e4fbba094124ecbd77c1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aa3fc2b2d4e4fbba094124ecbd77c1b" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 41741 } } peers { permanent_uuid: "4259d3d7a07445fb91bcf9cff27a444c" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 43415 } } peers { permanent_uuid: "08b740d5997d4132a29fd1a14ca86e80" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 34219 } }
I20260811 05:14:39.689375 17398 catalog_manager.cc:5495] T 0ee49582dba142f3bf8ed4023ce00c4f P 5aa3fc2b2d4e4fbba094124ecbd77c1b reported cstate change: term changed from 0 to 1, leader changed from <none> to 5aa3fc2b2d4e4fbba094124ecbd77c1b (127.28.143.129). New cstate: current_term: 1 leader_uuid: "5aa3fc2b2d4e4fbba094124ecbd77c1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5aa3fc2b2d4e4fbba094124ecbd77c1b" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 41741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4259d3d7a07445fb91bcf9cff27a444c" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 43415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08b740d5997d4132a29fd1a14ca86e80" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 34219 } health_report { overall_health: UNKNOWN } } }
I20260811 05:14:39.713896 18282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:39.707379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38192 (local address 127.28.143.129:41741)
0811 05:14:39.707429 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:39.707431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:39.707442 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:14:39.707511 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:39.707512 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:39.707538 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:39.707636 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:14:39.707817 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:39.708601 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:39.709923 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:39.710237 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:39.710303 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:39.710566 (+ 263us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:14:39.710730 (+ 164us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:14:39.710822 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 05:14:39.713599 (+ 2777us) server_negotiation.cc:299] Negotiation successful
0811 05:14:39.713637 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 05:14:39.716533 18166 raft_consensus.cc:1261] T 0ee49582dba142f3bf8ed4023ce00c4f P 08b740d5997d4132a29fd1a14ca86e80 [term 1 FOLLOWER]: Refusing update from remote peer 5aa3fc2b2d4e4fbba094124ecbd77c1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:14:39.716845 17949 raft_consensus.cc:1261] T 0ee49582dba142f3bf8ed4023ce00c4f P 4259d3d7a07445fb91bcf9cff27a444c [term 1 FOLLOWER]: Refusing update from remote peer 5aa3fc2b2d4e4fbba094124ecbd77c1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:14:39.717308 18308 consensus_queue.cc:1035] T 0ee49582dba142f3bf8ed4023ce00c4f P 5aa3fc2b2d4e4fbba094124ecbd77c1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4259d3d7a07445fb91bcf9cff27a444c" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 43415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:14:39.717649 18308 consensus_queue.cc:1035] T 0ee49582dba142f3bf8ed4023ce00c4f P 5aa3fc2b2d4e4fbba094124ecbd77c1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "08b740d5997d4132a29fd1a14ca86e80" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 34219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:14:39.726184 18314 mvcc.cc:205] Tried to move back new op lower bound from 7317197945715249152 to 7317197945603104768. Current Snapshot: MvccSnapshot[applied={T|T < 7317197945715249152}]
I20260811 05:14:39.726363 18316 mvcc.cc:205] Tried to move back new op lower bound from 7317197945715249152 to 7317197945603104768. Current Snapshot: MvccSnapshot[applied={T|T < 7317197945715249152 or (T in {7317197945715249152})}]
I20260811 05:14:39.727737 18319 mvcc.cc:205] Tried to move back new op lower bound from 7317197945715249152 to 7317197945603104768. Current Snapshot: MvccSnapshot[applied={T|T < 7317197945715249152}]
I20260811 05:14:39.739164 17398 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:14:39.739305 17398 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:14:39.740199 17398 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 021e199b03124f2991fc517df3d3cb7d: Sending DeleteTablet for 3 replicas of tablet 0ee49582dba142f3bf8ed4023ce00c4f
I20260811 05:14:39.740713 18121 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ee49582dba142f3bf8ed4023ce00c4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:14:39 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:38902
I20260811 05:14:39.741310 17913 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ee49582dba142f3bf8ed4023ce00c4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:14:39 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:50242
I20260811 05:14:39.741428 17659 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ee49582dba142f3bf8ed4023ce00c4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:14:39 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:38188
I20260811 05:14:39.742058 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 17538
I20260811 05:14:39.758378 18330 tablet_replica.cc:330] T 0ee49582dba142f3bf8ed4023ce00c4f P 08b740d5997d4132a29fd1a14ca86e80: stopping tablet replica
I20260811 05:14:39.758597 18330 raft_consensus.cc:2229] T 0ee49582dba142f3bf8ed4023ce00c4f P 08b740d5997d4132a29fd1a14ca86e80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:14:39.758754 18330 raft_consensus.cc:2258] T 0ee49582dba142f3bf8ed4023ce00c4f P 08b740d5997d4132a29fd1a14ca86e80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:14:39.759157 18331 tablet_replica.cc:330] T 0ee49582dba142f3bf8ed4023ce00c4f P 4259d3d7a07445fb91bcf9cff27a444c: stopping tablet replica
I20260811 05:14:39.759326 18331 raft_consensus.cc:2229] T 0ee49582dba142f3bf8ed4023ce00c4f P 4259d3d7a07445fb91bcf9cff27a444c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:14:39.759860 18331 raft_consensus.cc:2258] T 0ee49582dba142f3bf8ed4023ce00c4f P 4259d3d7a07445fb91bcf9cff27a444c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:14:39.759903 18330 ts_tablet_manager.cc:1905] T 0ee49582dba142f3bf8ed4023ce00c4f P 08b740d5997d4132a29fd1a14ca86e80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:14:39.760464 18331 ts_tablet_manager.cc:1905] T 0ee49582dba142f3bf8ed4023ce00c4f P 4259d3d7a07445fb91bcf9cff27a444c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:14:39.762018 18330 ts_tablet_manager.cc:1918] T 0ee49582dba142f3bf8ed4023ce00c4f P 08b740d5997d4132a29fd1a14ca86e80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:14:39.762090 18331 ts_tablet_manager.cc:1918] T 0ee49582dba142f3bf8ed4023ce00c4f P 4259d3d7a07445fb91bcf9cff27a444c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:14:39.762125 18330 log.cc:1196] T 0ee49582dba142f3bf8ed4023ce00c4f P 08b740d5997d4132a29fd1a14ca86e80: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-2/wal/wals/0ee49582dba142f3bf8ed4023ce00c4f
I20260811 05:14:39.762145 18331 log.cc:1196] T 0ee49582dba142f3bf8ed4023ce00c4f P 4259d3d7a07445fb91bcf9cff27a444c: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425159060450-29246-0/minicluster-data/ts-1/wal/wals/0ee49582dba142f3bf8ed4023ce00c4f
I20260811 05:14:39.762398 18331 ts_tablet_manager.cc:1939] T 0ee49582dba142f3bf8ed4023ce00c4f P 4259d3d7a07445fb91bcf9cff27a444c: Deleting consensus metadata
I20260811 05:14:39.762398 18330 ts_tablet_manager.cc:1939] T 0ee49582dba142f3bf8ed4023ce00c4f P 08b740d5997d4132a29fd1a14ca86e80: Deleting consensus metadata
I20260811 05:14:39.763154 17370 catalog_manager.cc:4841] TS 08b740d5997d4132a29fd1a14ca86e80 (127.28.143.131:34219): tablet 0ee49582dba142f3bf8ed4023ce00c4f (table test-table [id=b3b7f561d7924805aa3c07d7268ce15b]) successfully deleted
W20260811 05:14:39.763236 17375 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.143.129:41741 (error 108)
I20260811 05:14:39.763880 17373 catalog_manager.cc:4841] TS 4259d3d7a07445fb91bcf9cff27a444c (127.28.143.130:43415): tablet 0ee49582dba142f3bf8ed4023ce00c4f (table test-table [id=b3b7f561d7924805aa3c07d7268ce15b]) successfully deleted
I20260811 05:14:39.764556 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 17800
I20260811 05:14:39.773178 18283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:39.769049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:41741 (local address 127.0.0.1:38206)
0811 05:14:39.773013 (+ 3964us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:39.773123 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.129:41741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3913}
W20260811 05:14:39.773524 17375 catalog_manager.cc:4584] TS 5aa3fc2b2d4e4fbba094124ecbd77c1b (127.28.143.129:41741): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0ee49582dba142f3bf8ed4023ce00c4f: Network error: Client connection negotiation failed: client connection to 127.28.143.129:41741: connect: Connection refused (error 111)
I20260811 05:14:39.774513 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 18034
I20260811 05:14:39.827852 18283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:39.827678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:41741 (local address 127.0.0.1:38210)
0811 05:14:39.827753 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:39.827798 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.129:41741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 05:14:39.853291 18283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:39.853124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:41741 (local address 127.0.0.1:38216)
0811 05:14:39.853199 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:39.853238 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.129:41741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 05:14:39.855345 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 17344
I20260811 05:14:39.876070 29246 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 05:14:40.232177 29246 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 05:14:40.232321 29246 mini_ranger.cc:66] Stopping Ranger...
I20260811 05:14:40.578421 29246 mini_ranger.cc:68] Stopped Ranger
2026-08-11 05:14:40.578 UTC [16641] LOG: received smart shutdown request
2026-08-11 05:14:40.583 UTC [16641] LOG: background worker "logical replication launcher" (PID 16657) exited with exit code 1
2026-08-11 05:14:40.583 UTC [16649] LOG: shutting down
2026-08-11 05:14:40.793 UTC [16641] LOG: database system is shut down
2026-08-11T05:14:40Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (56746 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:14:40 dist-test-slave-f7th krb5kdc[18351](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:14:40 dist-test-slave-f7th krb5kdc[18351](info): set up 2 sockets
Aug 11 05:14:40 dist-test-slave-f7th krb5kdc[18351](info): commencing operation
krb5kdc: starting...
W20260811 05:14:42.138777 29246 mini_kdc.cc:121] Time spent starting KDC: real 1.219s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Aug 11 05:14:42 dist-test-slave-f7th krb5kdc[18351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425282, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:14:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:14:42Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.28.143.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.28.143.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.28.143.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/rangeradmin_127.28.143.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.28.143.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/rangeradmin_127.28.143.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.28.143.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.28.143.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.28.143.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/rangerlookup_127.28.143.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.28.143.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/rangerlookup_127.28.143.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.28.143.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.28.143.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.28.143.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/HTTP_127.28.143.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.28.143.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/HTTP_127.28.143.148@KRBTEST.COM.keytab.
I20260811 05:14:42.352927 29246 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskY_TM7_/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/postgres -l logfile start
2026-08-11 05:14:45.178 UTC [18400] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 05:14:45.178 UTC [18400] LOG: listening on IPv4 address "127.28.143.148", port 39275
2026-08-11 05:14:45.183 UTC [18400] LOG: listening on Unix socket "/tmp/.s.PGSQL.39275"
2026-08-11 05:14:45.212 UTC [18404] LOG: database system was shut down at 2026-08-11 05:14:43 UTC
2026-08-11 05:14:45.217 UTC [18400] LOG: database system is ready to accept connections
I20260811 05:14:46.346261 29246 mini_postgres.cc:96] Postgres bound to 39275
2026-08-11 05:14:46.353 UTC [18422] FATAL: database "slave" does not exist
127.28.143.148:39275 - accepting connections
I20260811 05:14:46.354272 29246 mini_ranger.cc:161] Starting Ranger...
I20260811 05:14:46.371129 29246 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 05:14:46.991178 29246 mini_ranger.cc:87] Created ranger Postgres database
I20260811 05:14:46.991254 29246 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ranger-admin
2026-08-11 05:14:47,276 [I] DB FLAVOR :POSTGRES
2026-08-11 05:14:47,277 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 05:14:47,277 [I] Checking connection..
2026-08-11 05:14:47,277 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:39275/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 05:14:47,556 [I] Checking connection passed.
2026-08-11 05:14:47,556 [I] --------- Verifying version history table ---------
2026-08-11 05:14:47,557 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:39275/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 05:14:47,807 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 05:14:47,807 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:39275/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 05:14:48,092 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 05:14:48,093 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 05:14:48,093 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:39275/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 05:14:48.354 UTC [18522] WARNING: there is no transaction in progress
2026-08-11 05:14:48,363 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 05:14:48,364 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:39275/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 05:14:48,633 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 05:14:48,633 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 05:14:48,634 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:39275/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 05:14:48,884 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:39275/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 05:14:49,165 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:39275/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-f7th.c.gcp-upstream.internal','N') ;"
2026-08-11 05:14:49,423 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:39275/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 05:14:49,695 [I] Table x_portal_user does not exist in database ranger
2026-08-11 05:14:49,696 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:39275/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 05:14:49,974 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 05:14:49,974 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:39275/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 05:14:50,255 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:39275/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 05:14:50,511 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 05:14:50,511 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:39275/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 05:14:51.720 UTC [18714] WARNING: there is no transaction in progress
2026-08-11 05:14:51.740 UTC [18714] WARNING: there is no transaction in progress
2026-08-11 05:14:51.760 UTC [18714] WARNING: there is no transaction in progress
2026-08-11 05:14:51.777 UTC [18714] WARNING: there is no transaction in progress
2026-08-11 05:14:51.803 UTC [18714] WARNING: there is no transaction in progress
2026-08-11 05:14:51.824 UTC [18714] WARNING: there is no transaction in progress
2026-08-11 05:14:51.892 UTC [18714] WARNING: there is no transaction in progress
2026-08-11 05:14:51.900 UTC [18714] WARNING: there is no transaction in progress
2026-08-11 05:14:51.917 UTC [18714] WARNING: there is no transaction in progress
2026-08-11 05:14:51.930 UTC [18714] WARNING: there is no transaction in progress
2026-08-11 05:14:51.943 UTC [18714] WARNING: there is no transaction in progress
2026-08-11 05:14:51.963 UTC [18714] WARNING: there is no transaction in progress
2026-08-11 05:14:51.974 UTC [18714] WARNING: there is no transaction in progress
2026-08-11 05:14:51.982 UTC [18714] WARNING: there is no transaction in progress
2026-08-11 05:14:51.990 UTC [18714] WARNING: there is no transaction in progress
2026-08-11 05:14:51.998 UTC [18714] WARNING: there is no transaction in progress
2026-08-11 05:14:52,858 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 05:14:52,859 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:39275/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 05:14:53,097 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 05:14:53,098 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:39275/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 05:14:53,373 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 05:14:53,373 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:39275/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 05:14:53,647 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 05:14:53,647 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:39275/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 05:14:53,912 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:39275/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 05:14:54,193 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:39275/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-f7th.c.gcp-upstream.internal';"
2026-08-11 05:14:54,445 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 05:14:54,446 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:39275/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 05:14:54,727 [I] DB_PATCHES have already been applied
I20260811 05:14:54.732589 29246 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ranger-admin:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskY_TM7_/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 05:14:54.732692 29246 mini_ranger.cc:193] Using host: 127.28.143.148
I20260811 05:14:54.737022 29246 mini_ranger.cc:227] Ranger admin URL: http://127.28.143.148:54135
Aug 11, 2026 5:14:55 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 5:14:55 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 5:14:55 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 5:14:55 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 5:14:55 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 5:14:55 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 5:14:55 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 05:14:56.056380 29246 mini_ranger.cc:160] Time spent starting Ranger: real 9.702s user 0.000s sys 0.011s
Aug 11, 2026 5:15:18 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 5:15:18 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 11, 2026 5:15:18 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 5:15:18 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 5:15:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 5:15:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 5:15:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 5:15:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 5:15:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 5:15:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 5:15:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 5:15:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 5:15:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 5:15:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 5:15:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 5:15:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 5:15:18 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 5:15:19 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 05:15:19.917157 29246 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.28.143.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.28.143.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.28.143.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/rangerkms_127.28.143.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.28.143.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/rangerkms_127.28.143.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.28.143.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.28.143.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.28.143.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/HTTP_127.28.143.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.28.143.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/HTTP_127.28.143.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 05:15:20 dist-test-slave-f7th krb5kdc[18351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425320, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 05:15:20.029371 29246 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 05:15:20.040992 29246 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 05:15:21.031502 29246 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 05:15:21.031582 29246 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ranger-kms
I20260811 05:15:21.031595 29246 mini_ranger_kms.cc:227] Using postgres at 127.28.143.148:39275
2026-08-11 05:15:21,203 [I] DB FLAVOR :POSTGRES
2026-08-11 05:15:21,203 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 05:15:21,203 [I] Checking connection
2026-08-11 05:15:21,203 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:39275/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 05:15:21,498 [I] connection success
2026-08-11 05:15:21,499 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 05:15:21,499 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 05:15:21,499 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:39275/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 05:15:21,763 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 05:15:21,763 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 05:15:21,763 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 05:15:21,763 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.143.148:39275/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 05:15:22,046 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 05:15:22.585492 29246 mini_ranger_kms.cc:310] Created kms service
I20260811 05:15:22.817147 29246 mini_ranger_kms.cc:326] Created kudu user
I20260811 05:15:22.870548 29246 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 05:15:22 dist-test-slave-f7th krb5kdc[18351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425320, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.28.143.148@KRBTEST.COM
I20260811 05:15:23.353559 29246 mini_ranger_kms.cc:384] Added ranger policy
I20260811 05:15:23.353703 29246 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ranger-kms:/tmp/dist-test-taskY_TM7_/build/release/bin/postgresql.jar:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskY_TM7_/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 05:15:23.353721 29246 mini_ranger_kms.cc:240] Using host: 127.28.143.148
I20260811 05:15:23.358645 29246 mini_ranger_kms.cc:276] Ranger KMS PID: 19048
I20260811 05:15:23.358752 29246 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.28.143.148:59437
Aug 11, 2026 5:15:24 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 5:15:24 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 5:15:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 5:15:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 5:15:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 5:15:24 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 05:15:25 dist-test-slave-f7th krb5kdc[18351](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786425325, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.28.143.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:15:26.010766 29246 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.981s user 0.023s sys 0.131s
I20260811 05:15:26.010856 29246 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 05:15:26.010903 29246 mini_ranger_kms.cc:401] 127.28.143.148:59437/kms/v1/keys
Aug 11 05:15:26 dist-test-slave-f7th krb5kdc[18351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425325, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.28.143.148@KRBTEST.COM for HTTP/127.28.143.148@KRBTEST.COM
Aug 11 05:15:26 dist-test-slave-f7th krb5kdc[18351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425325, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.28.143.148@KRBTEST.COM for HTTP/127.28.143.148@KRBTEST.COM
Aug 11 05:15:29 dist-test-slave-f7th krb5kdc[18351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.28.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
I20260811 05:15:29.180476 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:34353
--webserver_interface=127.28.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:43383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:34353
--ranger_config_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.143.148:59437/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:15:29.273793 19150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:15:29.274034 19150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:15:29.274058 19150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:15:29.276502 19150 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:15:29.276598 19150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:15:29.276651 19150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:15:29.276672 19150 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:15:29.276686 19150 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:15:29.278870 19150 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:43383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/master-0/wal
--ranger_kms_url=127.28.143.148:59437/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:34353
--ranger_config_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:34353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19150
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:15:29.279165 19150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:15:29.279403 19150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:15:29.284591 19157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:29.284649 19160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:29.284713 19158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:29.285077 19150 server_base.cc:1034] running on GCE node
I20260811 05:15:29.285367 19150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:15:29.285724 19150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:15:29.286911 19150 hybrid_clock.cc:648] HybridClock initialized: now 1786425329286891 us; error 35 us; skew 500 ppm
Aug 11 05:15:29 dist-test-slave-f7th krb5kdc[18351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425329, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:15:29.290236 19150 init.cc:377] Logged in from keytab as kudu/127.28.143.190@KRBTEST.COM (short username kudu)
I20260811 05:15:29.290856 19150 webserver.cc:460] Webserver started at http://127.28.143.190:37413/ using document root <none> and password file <none>
I20260811 05:15:29.291124 19150 fs_manager.cc:359] Metadata directory not provided
I20260811 05:15:29.291177 19150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:15:29.291313 19150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 05:15:29 dist-test-slave-f7th krb5kdc[18351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425329, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for HTTP/127.28.143.148@KRBTEST.COM
I20260811 05:15:29.341871 19150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "fe432f74cc3e45e0a8d562b4653559e8"
format_stamp: "Formatted at 2026-08-11 05:15:29 on dist-test-slave-f7th"
server_key: "ecd1fad56b0aa54a890c827a3f0db40d"
server_key_iv: "589bc6478b8895cfb4188731ddd71412"
server_key_version: "kuduclusterkey@0"
I20260811 05:15:29.342303 19150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "fe432f74cc3e45e0a8d562b4653559e8"
format_stamp: "Formatted at 2026-08-11 05:15:29 on dist-test-slave-f7th"
server_key: "ecd1fad56b0aa54a890c827a3f0db40d"
server_key_iv: "589bc6478b8895cfb4188731ddd71412"
server_key_version: "kuduclusterkey@0"
I20260811 05:15:29.343799 19150 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 05:15:29.359982 19171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:29.360239 19150 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:15:29.360342 19150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "fe432f74cc3e45e0a8d562b4653559e8"
format_stamp: "Formatted at 2026-08-11 05:15:29 on dist-test-slave-f7th"
server_key: "ecd1fad56b0aa54a890c827a3f0db40d"
server_key_iv: "589bc6478b8895cfb4188731ddd71412"
server_key_version: "kuduclusterkey@0"
I20260811 05:15:29.360442 19150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:15:29.396081 19150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:15:29.398192 19150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:15:29.398367 19150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:15:29.405898 19150 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:34353
I20260811 05:15:29.405993 19258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:34353 every 8 connection(s)
I20260811 05:15:29.406418 19150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:15:29.407366 19261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:15:29.411046 19261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8: Bootstrap starting.
I20260811 05:15:29.411986 19261 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8: Neither blocks nor log segments found. Creating new log.
I20260811 05:15:29.412348 19261 log.cc:824] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8: Log is configured to *not* fsync() on all Append() calls
I20260811 05:15:29.412845 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 19150
I20260811 05:15:29.412951 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:15:29.413157 29246 external_mini_cluster.cc:1428] Setting key c6fbd0ff41208f60a326a85015279e27
I20260811 05:15:29.413369 19261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8: No bootstrap required, opened a new log
I20260811 05:15:29.417172 19261 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe432f74cc3e45e0a8d562b4653559e8" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 34353 } }
I20260811 05:15:29.417521 19261 raft_consensus.cc:374] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:15:29.417698 19261 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe432f74cc3e45e0a8d562b4653559e8, State: Initialized, Role: FOLLOWER
I20260811 05:15:29.417964 19261 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe432f74cc3e45e0a8d562b4653559e8" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 34353 } }
I20260811 05:15:29.418179 19261 raft_consensus.cc:388] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:15:29.418282 19261 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:15:29.418350 19261 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:15:29.419009 19261 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe432f74cc3e45e0a8d562b4653559e8" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 34353 } }
I20260811 05:15:29.419130 19261 leader_election.cc:302] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe432f74cc3e45e0a8d562b4653559e8; no voters:
I20260811 05:15:29.419363 19261 leader_election.cc:288] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:15:29.419461 19268 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:15:29.419664 19268 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8 [term 1 LEADER]: Becoming Leader. State: Replica: fe432f74cc3e45e0a8d562b4653559e8, State: Running, Role: LEADER
I20260811 05:15:29.419761 19261 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:15:29.419801 19268 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe432f74cc3e45e0a8d562b4653559e8" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 34353 } }
I20260811 05:15:29.420392 19271 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fe432f74cc3e45e0a8d562b4653559e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe432f74cc3e45e0a8d562b4653559e8" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 34353 } } }
I20260811 05:15:29.420487 19271 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8 [sys.catalog]: This master's current role is: LEADER
I20260811 05:15:29.420717 19269 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe432f74cc3e45e0a8d562b4653559e8. Latest consensus state: current_term: 1 leader_uuid: "fe432f74cc3e45e0a8d562b4653559e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe432f74cc3e45e0a8d562b4653559e8" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 34353 } } }
I20260811 05:15:29.420776 19269 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8 [sys.catalog]: This master's current role is: LEADER
I20260811 05:15:29.421387 19261 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Aug 11 05:15:29 dist-test-slave-f7th krb5kdc[18351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:15:29.425239 19287 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:15:29.426057 19287 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:15:29.428390 19287 catalog_manager.cc:1347] Generated new cluster ID: 1438951f256c45b8a90d2ac9ace7fc13
I20260811 05:15:29.428524 19287 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:15:29.429553 19265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:29.413818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50614 (local address 127.28.143.190:34353)
0811 05:15:29.414019 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:29.414031 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:29.414068 (+ 37us) server_negotiation.cc:407] Connection header received
0811 05:15:29.415002 (+ 934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:29.415015 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:29.415096 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:29.415272 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:29.416490 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:29.418038 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:29.419234 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:29.421048 (+ 1814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:29.421169 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:29.424678 (+ 3509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:29.424742 (+ 64us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:29.424743 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:29.424849 (+ 106us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:29.427213 (+ 2364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:29.427689 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:29.427693 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:29.427694 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:29.427819 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:29.428042 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:29.428043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:29.428043 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:29.428401 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:29.428562 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:29.429156 (+ 594us) server_negotiation.cc:299] Negotiation successful
0811 05:15:29.429321 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":86,"threads_started":1}
I20260811 05:15:29.470916 19287 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:15:29.471807 19287 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:15:29.481563 19287 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8: Generated new TSK 0
I20260811 05:15:29.481930 19287 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.143.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab.
Entry for principal kudu/127.28.143.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab.
I20260811 05:15:30.211761 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.143.148:59437/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.129:0
--local_ip_for_outbound_sockets=127.28.143.129
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:34353
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:43383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:15:30.324401 19350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:15:30.324664 19350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:15:30.324700 19350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:15:30.326004 19350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:15:30.326056 19350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:15:30.326083 19350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.129
I20260811 05:15:30.327744 19350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:43383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.28.143.148:59437/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:34353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.129
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:15:30.327997 19350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:15:30.328258 19350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:15:30.335112 19357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:30.335310 19358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:30.335096 19361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:30.335657 19350 server_base.cc:1034] running on GCE node
I20260811 05:15:30.335879 19350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:15:30.336187 19350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:15:30.337354 19350 hybrid_clock.cc:648] HybridClock initialized: now 1786425330337331 us; error 38 us; skew 500 ppm
Aug 11 05:15:30 dist-test-slave-f7th krb5kdc[18351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425330, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:15:30.343736 19350 init.cc:377] Logged in from keytab as kudu/127.28.143.129@KRBTEST.COM (short username kudu)
I20260811 05:15:30.344435 19350 webserver.cc:460] Webserver started at http://127.28.143.129:45049/ using document root <none> and password file <none>
I20260811 05:15:30.344779 19350 fs_manager.cc:359] Metadata directory not provided
I20260811 05:15:30.344861 19350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:15:30.345013 19350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 05:15:30 dist-test-slave-f7th krb5kdc[18351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425330, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for HTTP/127.28.143.148@KRBTEST.COM
I20260811 05:15:30.361557 19350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-0/data/instance:
uuid: "a611f645ffae454a8ec3292f0e20d1b7"
format_stamp: "Formatted at 2026-08-11 05:15:30 on dist-test-slave-f7th"
server_key: "3ca10d748cc11a1ecbbf3165a4defb5c"
server_key_iv: "9c35f34341b51079baefc8b785f36895"
server_key_version: "kuduclusterkey@0"
I20260811 05:15:30.362113 19350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance:
uuid: "a611f645ffae454a8ec3292f0e20d1b7"
format_stamp: "Formatted at 2026-08-11 05:15:30 on dist-test-slave-f7th"
server_key: "3ca10d748cc11a1ecbbf3165a4defb5c"
server_key_iv: "9c35f34341b51079baefc8b785f36895"
server_key_version: "kuduclusterkey@0"
I20260811 05:15:30.370177 19350 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.001s sys 0.000s
I20260811 05:15:30.389772 19375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:30.390061 19350 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:15:30.390175 19350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-0/wal
uuid: "a611f645ffae454a8ec3292f0e20d1b7"
format_stamp: "Formatted at 2026-08-11 05:15:30 on dist-test-slave-f7th"
server_key: "3ca10d748cc11a1ecbbf3165a4defb5c"
server_key_iv: "9c35f34341b51079baefc8b785f36895"
server_key_version: "kuduclusterkey@0"
I20260811 05:15:30.390272 19350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:15:30.410430 19350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:15:30.413111 19350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:15:30.413285 19350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:15:30.413522 19350 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:15:30.413882 19350 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:15:30.413920 19350 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:30.413952 19350 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:15:30.413977 19350 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:30.421593 19350 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.129:39447
I20260811 05:15:30.422050 19350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
I20260811 05:15:30.426628 19574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.129:39447 every 8 connection(s)
Aug 11 05:15:30 dist-test-slave-f7th krb5kdc[18351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425330, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:15:30.432924 19577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:30.422898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:34353 (local address 127.28.143.129:56059)
0811 05:15:30.423094 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:30.423141 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:30.423640 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:30.424041 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:30.424045 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:30.424251 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:15:30.424458 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:30.424463 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:30.425391 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:30.425392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:30.426396 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:30.426400 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:30.426886 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:30.426887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:30.427040 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:30.427600 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:15:30.427609 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:15:30.428965 (+ 1356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:15:30.431021 (+ 2056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:30.431044 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:30.431047 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:30.431402 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:30.431653 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:30.431653 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:30.431654 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:30.431810 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:30.432138 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:15:30.432146 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:15:30.432340 (+ 194us) client_negotiation.cc:769] Sending connection context
0811 05:15:30.432441 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 05:15:30.432655 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":80,"threads_started":1}
I20260811 05:15:30.433844 19581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:30.422900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:56059 (local address 127.28.143.190:34353)
0811 05:15:30.423804 (+ 904us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:30.423806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:30.423815 (+ 9us) server_negotiation.cc:407] Connection header received
0811 05:15:30.423860 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:30.423861 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:30.423874 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:30.423940 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:30.424567 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:30.425285 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:30.426515 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:30.426747 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:30.427268 (+ 521us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:30.429080 (+ 1812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:30.429082 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:30.429082 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:30.429090 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:30.430902 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:30.431508 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:30.431509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:30.431509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:30.431580 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:30.431886 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:30.431887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:30.431887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:30.432056 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:30.433568 (+ 1512us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:30.433618 (+ 50us) server_negotiation.cc:299] Negotiation successful
0811 05:15:30.433726 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":868,"thread_start_us":100,"threads_started":1}
I20260811 05:15:30.434489 19575 heartbeater.cc:347] Connected to a master server at 127.28.143.190:34353
I20260811 05:15:30.434604 19575 heartbeater.cc:464] Registering TS with master...
I20260811 05:15:30.434834 19575 heartbeater.cc:511] Master 127.28.143.190:34353 requested a full tablet report, sending...
I20260811 05:15:30.435518 19197 ts_manager.cc:194] Registered new tserver with Master: a611f645ffae454a8ec3292f0e20d1b7 (127.28.143.129:39447)
I20260811 05:15:30.436467 19197 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.129@KRBTEST.COM'} at 127.28.143.129:56059
I20260811 05:15:30.440843 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 19350
I20260811 05:15:30.440949 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance
I20260811 05:15:30.441143 29246 external_mini_cluster.cc:1428] Setting key 168b275ea6eb3034e1951b4f8ef4d176
WARNING: no policy specified for kudu/127.28.143.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab.
Entry for principal kudu/127.28.143.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab.
I20260811 05:15:30.476228 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.143.148:59437/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.130:0
--local_ip_for_outbound_sockets=127.28.143.130
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:34353
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:43383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 05:15:30 dist-test-slave-f7th krb5kdc[18351](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786425330, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 05:15:30.583066 19587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:15:30.583443 19587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:15:30.583552 19587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:15:30.585391 19587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:15:30.585636 19587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:15:30.585744 19587 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.130
I20260811 05:15:30.588174 19587 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:43383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.28.143.148:59437/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:34353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19587
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.130
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:15:30.588804 19587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:15:30.589195 19587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:15:30.596323 19597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:30.596833 19595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:30.597131 19599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:30.598497 19587 server_base.cc:1034] running on GCE node
I20260811 05:15:30.598754 19587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:15:30.599072 19587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:15:30.602355 19587 hybrid_clock.cc:648] HybridClock initialized: now 1786425330602322 us; error 45 us; skew 500 ppm
Aug 11 05:15:30 dist-test-slave-f7th krb5kdc[18351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425330, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:15:30.606156 19587 init.cc:377] Logged in from keytab as kudu/127.28.143.130@KRBTEST.COM (short username kudu)
I20260811 05:15:30.606873 19587 webserver.cc:460] Webserver started at http://127.28.143.130:42525/ using document root <none> and password file <none>
I20260811 05:15:30.607151 19587 fs_manager.cc:359] Metadata directory not provided
I20260811 05:15:30.607213 19587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:15:30.607358 19587 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 05:15:30 dist-test-slave-f7th krb5kdc[18351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425330, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for HTTP/127.28.143.148@KRBTEST.COM
I20260811 05:15:30.627698 19587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-1/data/instance:
uuid: "602c82eac0e346bfbf403636010c9077"
format_stamp: "Formatted at 2026-08-11 05:15:30 on dist-test-slave-f7th"
server_key: "50d5c2bf1c8dc37e2ef27c324532ef9d"
server_key_iv: "4c0cf8946ae27786f727dee077de78d4"
server_key_version: "kuduclusterkey@0"
I20260811 05:15:30.628165 19587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance:
uuid: "602c82eac0e346bfbf403636010c9077"
format_stamp: "Formatted at 2026-08-11 05:15:30 on dist-test-slave-f7th"
server_key: "50d5c2bf1c8dc37e2ef27c324532ef9d"
server_key_iv: "4c0cf8946ae27786f727dee077de78d4"
server_key_version: "kuduclusterkey@0"
I20260811 05:15:30.630321 19587 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260811 05:15:30.654362 19610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:30.654645 19587 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.001s sys 0.001s
I20260811 05:15:30.654747 19587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-1/wal
uuid: "602c82eac0e346bfbf403636010c9077"
format_stamp: "Formatted at 2026-08-11 05:15:30 on dist-test-slave-f7th"
server_key: "50d5c2bf1c8dc37e2ef27c324532ef9d"
server_key_iv: "4c0cf8946ae27786f727dee077de78d4"
server_key_version: "kuduclusterkey@0"
I20260811 05:15:30.654843 19587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:15:30.684418 19587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:15:30.687029 19587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:15:30.687258 19587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:15:30.687556 19587 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:15:30.688047 19587 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:15:30.688143 19587 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:30.688220 19587 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:15:30.688282 19587 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:30.696484 19587 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.130:45993
I20260811 05:15:30.696727 19813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.130:45993 every 8 connection(s)
I20260811 05:15:30.697089 19587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
Aug 11 05:15:30 dist-test-slave-f7th krb5kdc[18351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425330, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:15:30.706025 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 19587
I20260811 05:15:30.706130 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance
I20260811 05:15:30.706328 29246 external_mini_cluster.cc:1428] Setting key 7affe89536a7e95404d856186f18c5b7
I20260811 05:15:30.708878 19581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:30.698129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:41997 (local address 127.28.143.190:34353)
0811 05:15:30.698183 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:30.698185 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:30.698421 (+ 236us) server_negotiation.cc:407] Connection header received
0811 05:15:30.699106 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:30.699108 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:30.699124 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:30.699153 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:30.700173 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:30.700933 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:30.702009 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:30.702320 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:30.702407 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:30.705300 (+ 2893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:30.705302 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:30.705302 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:30.705313 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:30.707015 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:30.707612 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:30.707614 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:30.707614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:30.707685 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:30.708005 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:30.708006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:30.708007 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:30.708171 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:30.708247 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:30.708674 (+ 427us) server_negotiation.cc:299] Negotiation successful
0811 05:15:30.708784 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 05:15:30.709192 19817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:30.698104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:34353 (local address 127.28.143.130:41997)
0811 05:15:30.698283 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:30.698334 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:30.698984 (+ 650us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:30.699334 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:30.699338 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:30.699724 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:15:30.700035 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:30.700042 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:30.701051 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:30.701052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:30.701893 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:30.701896 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:30.702450 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:30.702452 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:30.702626 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:30.703309 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:15:30.703321 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:15:30.705148 (+ 1827us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:15:30.707141 (+ 1993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:30.707153 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:30.707156 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:30.707503 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:30.707764 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:30.707765 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:30.707765 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:30.707929 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:30.708255 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:15:30.708263 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:15:30.708451 (+ 188us) client_negotiation.cc:769] Sending connection context
0811 05:15:30.708604 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 05:15:30.708804 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":95,"threads_started":1}
I20260811 05:15:30.710006 19814 heartbeater.cc:347] Connected to a master server at 127.28.143.190:34353
I20260811 05:15:30.710139 19814 heartbeater.cc:464] Registering TS with master...
I20260811 05:15:30.710368 19814 heartbeater.cc:511] Master 127.28.143.190:34353 requested a full tablet report, sending...
I20260811 05:15:30.710866 19197 ts_manager.cc:194] Registered new tserver with Master: 602c82eac0e346bfbf403636010c9077 (127.28.143.130:45993)
I20260811 05:15:30.711259 19197 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.130@KRBTEST.COM'} at 127.28.143.130:41997
WARNING: no policy specified for kudu/127.28.143.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab.
Entry for principal kudu/127.28.143.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab.
I20260811 05:15:30.740006 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.143.148:59437/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.131:0
--local_ip_for_outbound_sockets=127.28.143.131
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:34353
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:43383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:15:30.864971 19821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:15:30.865237 19821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:15:30.865264 19821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:15:30.866562 19821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:15:30.866617 19821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:15:30.866647 19821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.131
I20260811 05:15:30.868346 19821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:43383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.28.143.148:59437/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:34353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19821
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.131
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:15:30.868654 19821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:15:30.868928 19821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:15:30.874239 19830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:30.874401 19833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:30.874570 19831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:30.875073 19821 server_base.cc:1034] running on GCE node
I20260811 05:15:30.875303 19821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:15:30.875576 19821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:15:30.876696 19821 hybrid_clock.cc:648] HybridClock initialized: now 1786425330876678 us; error 36 us; skew 500 ppm
Aug 11 05:15:30 dist-test-slave-f7th krb5kdc[18351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425330, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:15:30.879881 19821 init.cc:377] Logged in from keytab as kudu/127.28.143.131@KRBTEST.COM (short username kudu)
I20260811 05:15:30.880509 19821 webserver.cc:460] Webserver started at http://127.28.143.131:33909/ using document root <none> and password file <none>
I20260811 05:15:30.880831 19821 fs_manager.cc:359] Metadata directory not provided
I20260811 05:15:30.880935 19821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:15:30.881105 19821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 05:15:30 dist-test-slave-f7th krb5kdc[18351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425330, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for HTTP/127.28.143.148@KRBTEST.COM
I20260811 05:15:30.896826 19821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-2/data/instance:
uuid: "e3c4f1facc964e7bb58e28804425d1d2"
format_stamp: "Formatted at 2026-08-11 05:15:30 on dist-test-slave-f7th"
server_key: "ba8bc9938961f8fc741ce450bc2cefef"
server_key_iv: "e2a659be455419941c3e3dfc38195ef1"
server_key_version: "kuduclusterkey@0"
I20260811 05:15:30.897380 19821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance:
uuid: "e3c4f1facc964e7bb58e28804425d1d2"
format_stamp: "Formatted at 2026-08-11 05:15:30 on dist-test-slave-f7th"
server_key: "ba8bc9938961f8fc741ce450bc2cefef"
server_key_iv: "e2a659be455419941c3e3dfc38195ef1"
server_key_version: "kuduclusterkey@0"
I20260811 05:15:30.909121 19821 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.002s sys 0.000s
I20260811 05:15:30.940428 19848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:30.940786 19821 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260811 05:15:30.940908 19821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-2/wal
uuid: "e3c4f1facc964e7bb58e28804425d1d2"
format_stamp: "Formatted at 2026-08-11 05:15:30 on dist-test-slave-f7th"
server_key: "ba8bc9938961f8fc741ce450bc2cefef"
server_key_iv: "e2a659be455419941c3e3dfc38195ef1"
server_key_version: "kuduclusterkey@0"
I20260811 05:15:30.941025 19821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:15:30.958817 19821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:15:30.961782 19821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:15:30.962070 19821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:15:30.962396 19821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:15:30.962903 19821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:15:30.963003 19821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:30.963088 19821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:15:30.963148 19821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:30.970861 19821 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.131:43961
I20260811 05:15:30.971807 19821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
I20260811 05:15:30.974504 20063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.131:43961 every 8 connection(s)
I20260811 05:15:30.979229 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 19821
I20260811 05:15:30.979341 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance
I20260811 05:15:30.979554 29246 external_mini_cluster.cc:1428] Setting key 90a1e3b9a34bd2d65e36ce7a9606c5c5
Aug 11 05:15:30 dist-test-slave-f7th krb5kdc[18351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425330, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:15:30.989216 19581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:30.975544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:52263 (local address 127.28.143.190:34353)
0811 05:15:30.975596 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:30.975598 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:30.975974 (+ 376us) server_negotiation.cc:407] Connection header received
0811 05:15:30.976773 (+ 799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:30.976775 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:30.976790 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:30.976821 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:30.977900 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:30.978695 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:30.982062 (+ 3367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:30.982360 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:30.982413 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:30.985454 (+ 3041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:30.985456 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:30.985457 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:30.985467 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:30.987252 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:30.987907 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:30.987909 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:30.987910 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:30.987982 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:30.988318 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:30.988319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:30.988320 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:30.988475 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:30.988526 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:30.989001 (+ 475us) server_negotiation.cc:299] Negotiation successful
0811 05:15:30.989103 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 05:15:30.990096 20071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:30.975218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:34353 (local address 127.28.143.131:52263)
0811 05:15:30.975809 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:30.975861 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:30.976587 (+ 726us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:30.977022 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:30.977027 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:30.977461 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:15:30.977777 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:30.977784 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:30.978831 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:30.978832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:30.981932 (+ 3100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:30.981938 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:30.982558 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:30.982559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:30.982718 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:30.983404 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:15:30.983416 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:15:30.985310 (+ 1894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:15:30.987379 (+ 2069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:30.987392 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:30.987397 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:30.987798 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:30.988056 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:30.988057 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:30.988057 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:30.988238 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:30.988575 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:15:30.988584 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:15:30.988821 (+ 237us) client_negotiation.cc:769] Sending connection context
0811 05:15:30.989623 (+ 802us) client_negotiation.cc:240] Negotiation successful
0811 05:15:30.989833 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":93,"threads_started":1}
I20260811 05:15:30.991863 20068 heartbeater.cc:347] Connected to a master server at 127.28.143.190:34353
I20260811 05:15:30.991983 20068 heartbeater.cc:464] Registering TS with master...
I20260811 05:15:30.992208 20068 heartbeater.cc:511] Master 127.28.143.190:34353 requested a full tablet report, sending...
I20260811 05:15:30.992736 19197 ts_manager.cc:194] Registered new tserver with Master: e3c4f1facc964e7bb58e28804425d1d2 (127.28.143.131:43961)
I20260811 05:15:30.993263 19197 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.131@KRBTEST.COM'} at 127.28.143.131:52263
I20260811 05:15:31.002897 29246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:15:31.003774 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 19350
I20260811 05:15:31.076283 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 19587
I20260811 05:15:31.116773 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 19821
I20260811 05:15:31.140035 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 19150
I20260811 05:15:31.197124 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:34353
--webserver_interface=127.28.143.190
--webserver_port=37413
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:43383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:34353
--ranger_config_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.143.148:59437/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.28.143.148:59437/kms with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:15:31.284403 20073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:15:31.284835 20073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:15:31.284963 20073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:15:31.286499 20073 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:15:31.286559 20073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:15:31.286578 20073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:15:31.286595 20073 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:15:31.286612 20073 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:15:31.288645 20073 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:43383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.28.143.148:59437/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:34353
--ranger_config_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:34353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=37413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:15:31.288942 20073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:15:31.289170 20073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:15:31.294284 20082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:31.294274 20084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:31.294274 20087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:31.294525 20073 server_base.cc:1034] running on GCE node
I20260811 05:15:31.294863 20073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:15:31.295164 20073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:15:31.296317 20073 hybrid_clock.cc:648] HybridClock initialized: now 1786425331296300 us; error 29 us; skew 500 ppm
Aug 11 05:15:31 dist-test-slave-f7th krb5kdc[18351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425331, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:15:31.299254 20073 init.cc:377] Logged in from keytab as kudu/127.28.143.190@KRBTEST.COM (short username kudu)
I20260811 05:15:31.299747 20073 webserver.cc:460] Webserver started at http://127.28.143.190:37413/ using document root <none> and password file <none>
I20260811 05:15:31.299961 20073 fs_manager.cc:359] Metadata directory not provided
I20260811 05:15:31.300026 20073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 05:15:31 dist-test-slave-f7th krb5kdc[18351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425331, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for HTTP/127.28.143.148@KRBTEST.COM
I20260811 05:15:31.885136 20073 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:15:31.885897 20100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:31.886102 20073 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:15:31.886206 20073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "fe432f74cc3e45e0a8d562b4653559e8"
format_stamp: "Formatted at 2026-08-11 05:15:29 on dist-test-slave-f7th"
server_key: "ecd1fad56b0aa54a890c827a3f0db40d"
server_key_iv: "589bc6478b8895cfb4188731ddd71412"
server_key_version: "kuduclusterkey@0"
I20260811 05:15:31.886693 20073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:15:31.905686 20073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:15:31.907542 20073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:15:31.907665 20073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:15:31.912060 20073 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:34353
I20260811 05:15:31.912113 20193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:34353 every 8 connection(s)
I20260811 05:15:31.912525 20073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:15:31.914029 20195 sys_catalog.cc:263] Verifying existing consensus state
I20260811 05:15:31.914763 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 20073
I20260811 05:15:31.914825 20195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8: Bootstrap starting.
I20260811 05:15:31.915026 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.143.148:59437/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.129:39447
--local_ip_for_outbound_sockets=127.28.143.129
--tserver_master_addrs=127.28.143.190:34353
--webserver_port=45049
--webserver_interface=127.28.143.129
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:43383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.28.143.148:59437/kms with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:15:31.923619 20195 log.cc:824] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8: Log is configured to *not* fsync() on all Append() calls
I20260811 05:15:31.925593 20195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 05:15:31.925879 20195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8: Bootstrap complete.
I20260811 05:15:31.928236 20195 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe432f74cc3e45e0a8d562b4653559e8" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 34353 } }
I20260811 05:15:31.928491 20195 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe432f74cc3e45e0a8d562b4653559e8, State: Initialized, Role: FOLLOWER
I20260811 05:15:31.928699 20195 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe432f74cc3e45e0a8d562b4653559e8" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 34353 } }
I20260811 05:15:31.928783 20195 raft_consensus.cc:388] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:15:31.928839 20195 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:15:31.928890 20195 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8 [term 1 FOLLOWER]: Advancing to term 2
I20260811 05:15:31.929961 20195 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe432f74cc3e45e0a8d562b4653559e8" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 34353 } }
I20260811 05:15:31.930074 20195 leader_election.cc:302] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe432f74cc3e45e0a8d562b4653559e8; no voters:
I20260811 05:15:31.930264 20195 leader_election.cc:288] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 05:15:31.930316 20204 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 05:15:31.930455 20204 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8 [term 2 LEADER]: Becoming Leader. State: Replica: fe432f74cc3e45e0a8d562b4653559e8, State: Running, Role: LEADER
I20260811 05:15:31.930590 20195 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:15:31.930636 20204 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe432f74cc3e45e0a8d562b4653559e8" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 34353 } }
I20260811 05:15:31.931052 20207 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe432f74cc3e45e0a8d562b4653559e8. Latest consensus state: current_term: 2 leader_uuid: "fe432f74cc3e45e0a8d562b4653559e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe432f74cc3e45e0a8d562b4653559e8" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 34353 } } }
I20260811 05:15:31.931137 20207 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8 [sys.catalog]: This master's current role is: LEADER
I20260811 05:15:31.931479 20214 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:15:31.931640 20206 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "fe432f74cc3e45e0a8d562b4653559e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe432f74cc3e45e0a8d562b4653559e8" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 34353 } } }
I20260811 05:15:31.931715 20206 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8 [sys.catalog]: This master's current role is: LEADER
I20260811 05:15:31.932055 20214 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:15:31.932243 20214 catalog_manager.cc:1259] Loaded cluster ID: 1438951f256c45b8a90d2ac9ace7fc13
I20260811 05:15:31.932317 20214 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:15:31.932443 20195 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 05:15:31.933672 20214 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:15:31.933890 20214 catalog_manager.cc:5879] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8: Loaded TSK: 0
I20260811 05:15:31.934128 20214 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 05:15:32.030735 20201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:15:32.031047 20201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:15:32.031088 20201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:15:32.033077 20201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:15:32.033141 20201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:15:32.033185 20201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.129
I20260811 05:15:32.035642 20201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:43383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.28.143.148:59437/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.129:39447
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.143.129
--webserver_port=45049
--tserver_master_addrs=127.28.143.190:34353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.129
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:15:32.036051 20201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:15:32.036387 20201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:15:32.043274 20275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:32.043272 20273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:32.043290 20272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:32.044075 20201 server_base.cc:1034] running on GCE node
I20260811 05:15:32.044318 20201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:15:32.044657 20201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:15:32.045845 20201 hybrid_clock.cc:648] HybridClock initialized: now 1786425332045830 us; error 30 us; skew 500 ppm
Aug 11 05:15:32 dist-test-slave-f7th krb5kdc[18351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425332, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:15:32.049290 20201 init.cc:377] Logged in from keytab as kudu/127.28.143.129@KRBTEST.COM (short username kudu)
I20260811 05:15:32.049852 20201 webserver.cc:460] Webserver started at http://127.28.143.129:45049/ using document root <none> and password file <none>
I20260811 05:15:32.050139 20201 fs_manager.cc:359] Metadata directory not provided
I20260811 05:15:32.050227 20201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 05:15:32 dist-test-slave-f7th krb5kdc[18351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425332, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for HTTP/127.28.143.148@KRBTEST.COM
I20260811 05:15:32.069974 20201 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260811 05:15:32.077354 20294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:32.077625 20201 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.000s sys 0.000s
I20260811 05:15:32.077775 20201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-0/wal
uuid: "a611f645ffae454a8ec3292f0e20d1b7"
format_stamp: "Formatted at 2026-08-11 05:15:30 on dist-test-slave-f7th"
server_key: "3ca10d748cc11a1ecbbf3165a4defb5c"
server_key_iv: "9c35f34341b51079baefc8b785f36895"
server_key_version: "kuduclusterkey@0"
I20260811 05:15:32.078315 20201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:15:32.096647 20201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:15:32.099251 20201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:15:32.099452 20201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:15:32.099768 20201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:15:32.100178 20201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:15:32.100224 20201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:32.100255 20201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:15:32.100272 20201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:32.108031 20201 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.129:39447
I20260811 05:15:32.108098 20501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.129:39447 every 8 connection(s)
I20260811 05:15:32.108565 20201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
I20260811 05:15:32.109952 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 20201
I20260811 05:15:32.110273 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.143.148:59437/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.130:45993
--local_ip_for_outbound_sockets=127.28.143.130
--tserver_master_addrs=127.28.143.190:34353
--webserver_port=42525
--webserver_interface=127.28.143.130
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:43383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.28.143.148:59437/kms with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 05:15:32 dist-test-slave-f7th krb5kdc[18351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425332, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:15:32.123436 20507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:32.109605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:34353 (local address 127.28.143.129:57189)
0811 05:15:32.109897 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:32.110099 (+ 202us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:32.110739 (+ 640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:32.111481 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:32.111486 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:32.111728 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:15:32.112049 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:32.112055 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:32.113884 (+ 1829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:32.113885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:32.115082 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:32.115085 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:32.115990 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:32.115992 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:32.116164 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:32.116890 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:15:32.116903 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:15:32.118693 (+ 1790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:15:32.121268 (+ 2575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:32.121279 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:32.121283 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:32.121616 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:32.121956 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:32.121957 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:32.121958 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:32.122127 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:32.122698 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:15:32.122706 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:15:32.122873 (+ 167us) client_negotiation.cc:769] Sending connection context
0811 05:15:32.122968 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 05:15:32.123171 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":90,"threads_started":1}
I20260811 05:15:32.124272 20508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:32.109724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:57189 (local address 127.28.143.190:34353)
0811 05:15:32.110999 (+ 1275us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:32.111009 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:32.111052 (+ 43us) server_negotiation.cc:407] Connection header received
0811 05:15:32.111206 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:32.111212 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:32.111274 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:32.112323 (+ 1049us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:32.112432 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:32.113746 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:32.115203 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:32.115844 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:32.115969 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:32.118867 (+ 2898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:32.118874 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:32.118874 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:32.118891 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:32.121145 (+ 2254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:32.121728 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:32.121732 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:32.121732 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:32.121861 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:32.122220 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:32.122221 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:32.122222 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:32.122600 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:32.123793 (+ 1193us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:32.123885 (+ 92us) server_negotiation.cc:299] Negotiation successful
0811 05:15:32.124053 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1200,"thread_start_us":97,"threads_started":1}
I20260811 05:15:32.125047 20502 heartbeater.cc:347] Connected to a master server at 127.28.143.190:34353
I20260811 05:15:32.125164 20502 heartbeater.cc:464] Registering TS with master...
I20260811 05:15:32.125366 20502 heartbeater.cc:511] Master 127.28.143.190:34353 requested a full tablet report, sending...
W20260811 05:15:32.126432 20502 heartbeater.cc:644] Failed to heartbeat to 127.28.143.190:34353 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 05:15:32.126504 20502 heartbeater.cc:464] Registering TS with master...
I20260811 05:15:32.126567 20502 heartbeater.cc:511] Master 127.28.143.190:34353 requested a full tablet report, sending...
I20260811 05:15:32.127087 20502 heartbeater.cc:464] Registering TS with master...
I20260811 05:15:32.127166 20502 heartbeater.cc:511] Master 127.28.143.190:34353 requested a full tablet report, sending...
W20260811 05:15:32.127558 20502 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 05:15:32.221939 20509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:15:32.222271 20509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:15:32.222312 20509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:15:32.224175 20509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:15:32.224251 20509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:15:32.224294 20509 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.130
I20260811 05:15:32.226411 20509 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:43383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.28.143.148:59437/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.130:45993
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.143.130
--webserver_port=42525
--tserver_master_addrs=127.28.143.190:34353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.130
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:15:32.226697 20509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:15:32.226964 20509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:15:32.233837 20516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:32.234822 20521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:32.235112 20515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:32.235770 20509 server_base.cc:1034] running on GCE node
I20260811 05:15:32.236056 20509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:15:32.236377 20509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:15:32.251904 20509 hybrid_clock.cc:648] HybridClock initialized: now 1786425332251818 us; error 57 us; skew 500 ppm
Aug 11 05:15:32 dist-test-slave-f7th krb5kdc[18351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425332, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:15:32.254807 20509 init.cc:377] Logged in from keytab as kudu/127.28.143.130@KRBTEST.COM (short username kudu)
I20260811 05:15:32.255380 20509 webserver.cc:460] Webserver started at http://127.28.143.130:42525/ using document root <none> and password file <none>
I20260811 05:15:32.255734 20509 fs_manager.cc:359] Metadata directory not provided
I20260811 05:15:32.255834 20509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 05:15:32 dist-test-slave-f7th krb5kdc[18351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425332, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for HTTP/127.28.143.148@KRBTEST.COM
I20260811 05:15:32.280755 20509 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:15:32.281635 20535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:32.281855 20509 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:15:32.281965 20509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-1/wal
uuid: "602c82eac0e346bfbf403636010c9077"
format_stamp: "Formatted at 2026-08-11 05:15:30 on dist-test-slave-f7th"
server_key: "50d5c2bf1c8dc37e2ef27c324532ef9d"
server_key_iv: "4c0cf8946ae27786f727dee077de78d4"
server_key_version: "kuduclusterkey@0"
I20260811 05:15:32.282470 20509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:15:32.301162 20509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:15:32.303994 20509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:15:32.304250 20509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:15:32.304579 20509 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:15:32.305075 20509 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:15:32.305173 20509 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:32.305248 20509 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:15:32.305306 20509 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:32.313134 20509 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.130:45993
I20260811 05:15:32.313411 20724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.130:45993 every 8 connection(s)
I20260811 05:15:32.313766 20509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
I20260811 05:15:32.319507 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 20509
I20260811 05:15:32.319829 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.143.148:59437/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.131:43961
--local_ip_for_outbound_sockets=127.28.143.131
--tserver_master_addrs=127.28.143.190:34353
--webserver_port=33909
--webserver_interface=127.28.143.131
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:43383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.28.143.148:59437/kms with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 05:15:32 dist-test-slave-f7th krb5kdc[18351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425332, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:15:32.324313 20508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:32.314562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:43891 (local address 127.28.143.190:34353)
0811 05:15:32.314612 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:32.314614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:32.314926 (+ 312us) server_negotiation.cc:407] Connection header received
0811 05:15:32.315504 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:32.315505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:32.315524 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:32.315575 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:32.316137 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:32.316687 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:32.317756 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:32.318001 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:32.318073 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:32.321004 (+ 2931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:32.321007 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:32.321008 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:32.321020 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:32.322609 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:32.323191 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:32.323193 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:32.323194 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:32.323265 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:32.323516 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:32.323517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:32.323517 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:32.323663 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:32.323711 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:32.324110 (+ 399us) server_negotiation.cc:299] Negotiation successful
0811 05:15:32.324218 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 05:15:32.324478 20729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:32.314567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:34353 (local address 127.28.143.130:43891)
0811 05:15:32.314778 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:32.314823 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:32.315401 (+ 578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:32.315644 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:32.315648 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:32.315837 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:15:32.316036 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:32.316040 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:32.316795 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:32.316796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:32.317657 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:32.317660 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:32.318127 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:32.318129 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:32.318294 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:32.318906 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:15:32.318918 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:15:32.320833 (+ 1915us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:15:32.322707 (+ 1874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:32.322717 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:32.322721 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:32.323082 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:32.323327 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:32.323328 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:32.323328 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:32.323443 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:32.323761 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:15:32.323769 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:15:32.323966 (+ 197us) client_negotiation.cc:769] Sending connection context
0811 05:15:32.324070 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 05:15:32.324237 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":117,"threads_started":1}
I20260811 05:15:32.325057 20726 heartbeater.cc:347] Connected to a master server at 127.28.143.190:34353
I20260811 05:15:32.325165 20726 heartbeater.cc:464] Registering TS with master...
I20260811 05:15:32.325374 20726 heartbeater.cc:511] Master 127.28.143.190:34353 requested a full tablet report, sending...
W20260811 05:15:32.326057 20726 heartbeater.cc:644] Failed to heartbeat to 127.28.143.190:34353 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 05:15:32.326129 20726 heartbeater.cc:464] Registering TS with master...
I20260811 05:15:32.326202 20726 heartbeater.cc:511] Master 127.28.143.190:34353 requested a full tablet report, sending...
I20260811 05:15:32.326520 20726 heartbeater.cc:464] Registering TS with master...
I20260811 05:15:32.326606 20726 heartbeater.cc:511] Master 127.28.143.190:34353 requested a full tablet report, sending...
W20260811 05:15:32.327415 20726 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 05:15:32.440582 20730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:15:32.440867 20730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:15:32.440904 20730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:15:32.442234 20730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:15:32.442286 20730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:15:32.442325 20730 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.131
I20260811 05:15:32.443986 20730 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:43383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.28.143.148:59437/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.131:43961
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.143.131
--webserver_port=33909
--tserver_master_addrs=127.28.143.190:34353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20730
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.131
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:15:32.444257 20730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:15:32.444509 20730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:15:32.449661 20741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:32.449673 20745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:32.449770 20739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:32.450451 20730 server_base.cc:1034] running on GCE node
I20260811 05:15:32.450731 20730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:15:32.451002 20730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:15:32.452172 20730 hybrid_clock.cc:648] HybridClock initialized: now 1786425332452156 us; error 29 us; skew 500 ppm
Aug 11 05:15:32 dist-test-slave-f7th krb5kdc[18351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425332, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:15:32.455102 20730 init.cc:377] Logged in from keytab as kudu/127.28.143.131@KRBTEST.COM (short username kudu)
I20260811 05:15:32.455714 20730 webserver.cc:460] Webserver started at http://127.28.143.131:33909/ using document root <none> and password file <none>
I20260811 05:15:32.455999 20730 fs_manager.cc:359] Metadata directory not provided
I20260811 05:15:32.456079 20730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 05:15:32 dist-test-slave-f7th krb5kdc[18351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425332, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for HTTP/127.28.143.148@KRBTEST.COM
I20260811 05:15:32.471695 20730 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:15:32.472699 20759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:32.472942 20730 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:15:32.473047 20730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-2/wal
uuid: "e3c4f1facc964e7bb58e28804425d1d2"
format_stamp: "Formatted at 2026-08-11 05:15:30 on dist-test-slave-f7th"
server_key: "ba8bc9938961f8fc741ce450bc2cefef"
server_key_iv: "e2a659be455419941c3e3dfc38195ef1"
server_key_version: "kuduclusterkey@0"
I20260811 05:15:32.473554 20730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:15:32.506580 20730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:15:32.509521 20730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:15:32.509779 20730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:15:32.510068 20730 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:15:32.510543 20730 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:15:32.510655 20730 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:32.510746 20730 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:15:32.510800 20730 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:32.519196 20730 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.131:43961
I20260811 05:15:32.519295 20962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.131:43961 every 8 connection(s)
I20260811 05:15:32.520076 20730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
Aug 11 05:15:32 dist-test-slave-f7th krb5kdc[18351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425332, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:15:32.528883 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 20730
I20260811 05:15:32.532699 20508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:32.521343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:46693 (local address 127.28.143.190:34353)
0811 05:15:32.521400 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:32.521403 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:32.521631 (+ 228us) server_negotiation.cc:407] Connection header received
0811 05:15:32.522319 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:32.522320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:32.522333 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:32.522406 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:32.523179 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:32.523928 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:32.525502 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:32.525822 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:32.525905 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:32.529151 (+ 3246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:32.529153 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:32.529154 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:32.529163 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:32.530786 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:32.531391 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:32.531393 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:32.531393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:32.531464 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:32.531827 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:32.531828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:32.531828 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:32.531998 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:32.532049 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:32.532453 (+ 404us) server_negotiation.cc:299] Negotiation successful
0811 05:15:32.532552 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 05:15:32.532904 20967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:32.520995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:34353 (local address 127.28.143.131:46693)
0811 05:15:32.521472 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:32.521522 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:32.522183 (+ 661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:32.522499 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:32.522503 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:32.522755 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:15:32.523063 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:32.523071 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:32.524064 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:32.524065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:32.525388 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:32.525393 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:32.525951 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:32.525953 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:32.526112 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:32.526791 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:15:32.526802 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:15:32.528596 (+ 1794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:15:32.530883 (+ 2287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:32.530894 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:32.530897 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:32.531288 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:32.531541 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:32.531541 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:32.531542 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:32.531745 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:32.532099 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:15:32.532107 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:15:32.532306 (+ 199us) client_negotiation.cc:769] Sending connection context
0811 05:15:32.532427 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 05:15:32.532610 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":86,"threads_started":1}
I20260811 05:15:32.533530 20963 heartbeater.cc:347] Connected to a master server at 127.28.143.190:34353
I20260811 05:15:32.533645 20963 heartbeater.cc:464] Registering TS with master...
I20260811 05:15:32.533881 20963 heartbeater.cc:511] Master 127.28.143.190:34353 requested a full tablet report, sending...
W20260811 05:15:32.534559 20963 heartbeater.cc:644] Failed to heartbeat to 127.28.143.190:34353 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 05:15:32.534641 20963 heartbeater.cc:464] Registering TS with master...
I20260811 05:15:32.534701 20963 heartbeater.cc:511] Master 127.28.143.190:34353 requested a full tablet report, sending...
I20260811 05:15:32.535027 20963 heartbeater.cc:464] Registering TS with master...
I20260811 05:15:32.535108 20963 heartbeater.cc:511] Master 127.28.143.190:34353 requested a full tablet report, sending...
W20260811 05:15:32.535367 20963 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 05:15:32.546286 20972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:32.530006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46492 (local address 127.28.143.190:34353)
0811 05:15:32.530254 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:32.530256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:32.532374 (+ 2118us) server_negotiation.cc:407] Connection header received
0811 05:15:32.533139 (+ 765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:32.533140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:32.533153 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:32.533223 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:32.537532 (+ 4309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:32.538329 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:32.540699 (+ 2370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:32.541050 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:32.541121 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:32.542721 (+ 1600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:32.542723 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:32.542724 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:32.542732 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:32.544502 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:32.544946 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:32.544947 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:32.544948 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:32.545022 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:32.545220 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:32.545221 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:32.545221 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:32.545364 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:32.545552 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:32.546063 (+ 511us) server_negotiation.cc:299] Negotiation successful
0811 05:15:32.546165 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":88,"threads_started":1}
Aug 11 05:15:33 dist-test-slave-f7th krb5kdc[18351](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786425333, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:15:33.127782 20502 heartbeater.cc:464] Registering TS with master...
I20260811 05:15:33.127964 20502 heartbeater.cc:511] Master 127.28.143.190:34353 requested a full tablet report, sending...
I20260811 05:15:33.128528 20132 ts_manager.cc:194] Registered new tserver with Master: a611f645ffae454a8ec3292f0e20d1b7 (127.28.143.129:39447)
I20260811 05:15:33.129558 20132 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.129@KRBTEST.COM'} at 127.28.143.129:57189
I20260811 05:15:33.327633 20726 heartbeater.cc:464] Registering TS with master...
I20260811 05:15:33.327819 20726 heartbeater.cc:511] Master 127.28.143.190:34353 requested a full tablet report, sending...
I20260811 05:15:33.328325 20132 ts_manager.cc:194] Registered new tserver with Master: 602c82eac0e346bfbf403636010c9077 (127.28.143.130:45993)
I20260811 05:15:33.328891 20132 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.130@KRBTEST.COM'} at 127.28.143.130:43891
Aug 11 05:15:33 dist-test-slave-f7th krb5kdc[18351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425333, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for HTTP/127.28.143.148@KRBTEST.COM
I20260811 05:15:33.535544 20963 heartbeater.cc:464] Registering TS with master...
I20260811 05:15:33.535719 20963 heartbeater.cc:511] Master 127.28.143.190:34353 requested a full tablet report, sending...
I20260811 05:15:33.536056 20132 ts_manager.cc:194] Registered new tserver with Master: e3c4f1facc964e7bb58e28804425d1d2 (127.28.143.131:43961)
I20260811 05:15:33.536549 20132 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.131@KRBTEST.COM'} at 127.28.143.131:46693
I20260811 05:15:33.544072 29246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:15:33.564597 21010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:33.548952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46500 (local address 127.28.143.190:34353)
0811 05:15:33.553331 (+ 4379us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:33.553334 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:33.553345 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:15:33.553375 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:33.553376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:33.553392 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:33.553492 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:33.557063 (+ 3571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:33.557889 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:33.559017 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:33.559343 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:33.559401 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:33.561134 (+ 1733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:33.561136 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:33.561137 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:33.561148 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:33.563008 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:33.563436 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:33.563438 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:33.563438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:33.563503 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:33.563687 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:33.563688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:33.563688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:33.563832 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:33.563935 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:33.564393 (+ 458us) server_negotiation.cc:299] Negotiation successful
0811 05:15:33.564487 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4349,"thread_start_us":104,"threads_started":1}
I20260811 05:15:33.567471 20132 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46500:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:15:33.570276 20132 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:15:33.583794 21021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:33.575300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:45993 (local address 127.0.0.1:56510)
0811 05:15:33.577280 (+ 1980us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:33.577289 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:33.577327 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:33.578156 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:33.578157 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:33.578159 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:15:33.578217 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:33.578219 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:33.580600 (+ 2381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:33.580601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:33.582050 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:33.582056 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:33.583433 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:33.583435 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:33.583548 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:33.583565 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 05:15:33.583640 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 05:15:33.583698 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1964,"thread_start_us":238,"threads_started":1}
I20260811 05:15:33.584065 21030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:33.575742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56510 (local address 127.28.143.130:45993)
0811 05:15:33.576810 (+ 1068us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:33.576814 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:33.577379 (+ 565us) server_negotiation.cc:407] Connection header received
0811 05:15:33.577420 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:33.577422 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:33.577438 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:33.577519 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:15:33.578298 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:33.580455 (+ 2157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:33.582268 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:33.583166 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:33.583314 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:33.583390 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:33.583941 (+ 551us) server_negotiation.cc:299] Negotiation successful
0811 05:15:33.583989 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1028,"thread_start_us":164,"threads_started":1}
I20260811 05:15:33.585022 20617 tablet_service.cc:1467] Processing CreateTablet for tablet 8a4647064e714c348fb510c5fde0c887 (DEFAULT_TABLE table=test-table [id=940ede0223a440f88229e922acf5ad90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:15:33.585451 20617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a4647064e714c348fb510c5fde0c887. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:15:33.585899 21029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:33.574776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:39447 (local address 127.0.0.1:38770)
0811 05:15:33.576635 (+ 1859us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:33.576644 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:33.576696 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:33.577096 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:33.577100 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:33.577103 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:15:33.577180 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:33.577182 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:33.582368 (+ 5186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:33.582369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:33.584559 (+ 2190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:33.584563 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:33.585653 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:33.585654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:33.585720 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:33.585725 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 05:15:33.585770 (+ 45us) client_negotiation.cc:240] Negotiation successful
0811 05:15:33.585815 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1821,"thread_start_us":83,"threads_started":1}
I20260811 05:15:33.586086 21024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:33.574568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58890 (local address 127.28.143.131:43961)
0811 05:15:33.582289 (+ 7721us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:33.582295 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:33.582306 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:15:33.582348 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:33.582350 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:33.582371 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:33.582459 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:15:33.582695 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:33.583737 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:33.584916 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:33.585641 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:33.585707 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:33.585755 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:33.585964 (+ 209us) server_negotiation.cc:299] Negotiation successful
0811 05:15:33.586010 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7669,"thread_start_us":55,"threads_started":1}
I20260811 05:15:33.586212 21026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:33.575052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38770 (local address 127.28.143.129:39447)
0811 05:15:33.576558 (+ 1506us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:33.576563 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:33.576841 (+ 278us) server_negotiation.cc:407] Connection header received
0811 05:15:33.576884 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:33.576886 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:33.576905 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:33.576979 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:15:33.577613 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:33.582281 (+ 4668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:33.584680 (+ 2399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:33.585462 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:33.585549 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:33.585610 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:33.586089 (+ 479us) server_negotiation.cc:299] Negotiation successful
0811 05:15:33.586141 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1465,"thread_start_us":52,"threads_started":1}
I20260811 05:15:33.586237 21025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:33.574512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.131:43961 (local address 127.0.0.1:58890)
0811 05:15:33.576278 (+ 1766us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:33.576330 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:33.576401 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:33.582561 (+ 6160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:33.582562 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:33.582566 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:15:33.582614 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:33.582615 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:33.583823 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:33.583824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:33.584831 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:33.584833 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:33.585786 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:33.585787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:33.585835 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:33.585837 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 05:15:33.585880 (+ 43us) client_negotiation.cc:240] Negotiation successful
0811 05:15:33.585909 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1726,"thread_start_us":87,"threads_started":1}
I20260811 05:15:33.586943 20851 tablet_service.cc:1467] Processing CreateTablet for tablet 8a4647064e714c348fb510c5fde0c887 (DEFAULT_TABLE table=test-table [id=940ede0223a440f88229e922acf5ad90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:15:33.587265 20851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a4647064e714c348fb510c5fde0c887. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:15:33.589187 20378 tablet_service.cc:1467] Processing CreateTablet for tablet 8a4647064e714c348fb510c5fde0c887 (DEFAULT_TABLE table=test-table [id=940ede0223a440f88229e922acf5ad90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:15:33.589589 20378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a4647064e714c348fb510c5fde0c887. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:15:33.591253 21032 tablet_bootstrap.cc:492] T 8a4647064e714c348fb510c5fde0c887 P e3c4f1facc964e7bb58e28804425d1d2: Bootstrap starting.
I20260811 05:15:33.592097 21032 tablet_bootstrap.cc:654] T 8a4647064e714c348fb510c5fde0c887 P e3c4f1facc964e7bb58e28804425d1d2: Neither blocks nor log segments found. Creating new log.
I20260811 05:15:33.592487 21032 log.cc:824] T 8a4647064e714c348fb510c5fde0c887 P e3c4f1facc964e7bb58e28804425d1d2: Log is configured to *not* fsync() on all Append() calls
I20260811 05:15:33.593123 21031 tablet_bootstrap.cc:492] T 8a4647064e714c348fb510c5fde0c887 P 602c82eac0e346bfbf403636010c9077: Bootstrap starting.
I20260811 05:15:33.594434 21031 tablet_bootstrap.cc:654] T 8a4647064e714c348fb510c5fde0c887 P 602c82eac0e346bfbf403636010c9077: Neither blocks nor log segments found. Creating new log.
I20260811 05:15:33.594785 21031 log.cc:824] T 8a4647064e714c348fb510c5fde0c887 P 602c82eac0e346bfbf403636010c9077: Log is configured to *not* fsync() on all Append() calls
I20260811 05:15:33.595862 21032 tablet_bootstrap.cc:492] T 8a4647064e714c348fb510c5fde0c887 P e3c4f1facc964e7bb58e28804425d1d2: No bootstrap required, opened a new log
I20260811 05:15:33.595952 21032 ts_tablet_manager.cc:1397] T 8a4647064e714c348fb510c5fde0c887 P e3c4f1facc964e7bb58e28804425d1d2: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260811 05:15:33.598732 21032 raft_consensus.cc:348] T 8a4647064e714c348fb510c5fde0c887 P e3c4f1facc964e7bb58e28804425d1d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c4f1facc964e7bb58e28804425d1d2" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 43961 } } peers { permanent_uuid: "a611f645ffae454a8ec3292f0e20d1b7" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 39447 } } peers { permanent_uuid: "602c82eac0e346bfbf403636010c9077" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 45993 } }
I20260811 05:15:33.598964 21032 raft_consensus.cc:374] T 8a4647064e714c348fb510c5fde0c887 P e3c4f1facc964e7bb58e28804425d1d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:15:33.599000 21032 raft_consensus.cc:729] T 8a4647064e714c348fb510c5fde0c887 P e3c4f1facc964e7bb58e28804425d1d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3c4f1facc964e7bb58e28804425d1d2, State: Initialized, Role: FOLLOWER
I20260811 05:15:33.599189 21032 consensus_queue.cc:260] T 8a4647064e714c348fb510c5fde0c887 P e3c4f1facc964e7bb58e28804425d1d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c4f1facc964e7bb58e28804425d1d2" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 43961 } } peers { permanent_uuid: "a611f645ffae454a8ec3292f0e20d1b7" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 39447 } } peers { permanent_uuid: "602c82eac0e346bfbf403636010c9077" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 45993 } }
I20260811 05:15:33.599450 21035 tablet_bootstrap.cc:492] T 8a4647064e714c348fb510c5fde0c887 P a611f645ffae454a8ec3292f0e20d1b7: Bootstrap starting.
I20260811 05:15:33.599857 21032 ts_tablet_manager.cc:1428] T 8a4647064e714c348fb510c5fde0c887 P e3c4f1facc964e7bb58e28804425d1d2: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260811 05:15:33.600062 20963 heartbeater.cc:503] Master 127.28.143.190:34353 was elected leader, sending a full tablet report...
I20260811 05:15:33.600409 21035 tablet_bootstrap.cc:654] T 8a4647064e714c348fb510c5fde0c887 P a611f645ffae454a8ec3292f0e20d1b7: Neither blocks nor log segments found. Creating new log.
I20260811 05:15:33.601585 21035 log.cc:824] T 8a4647064e714c348fb510c5fde0c887 P a611f645ffae454a8ec3292f0e20d1b7: Log is configured to *not* fsync() on all Append() calls
I20260811 05:15:33.602754 21035 tablet_bootstrap.cc:492] T 8a4647064e714c348fb510c5fde0c887 P a611f645ffae454a8ec3292f0e20d1b7: No bootstrap required, opened a new log
I20260811 05:15:33.602762 21031 tablet_bootstrap.cc:492] T 8a4647064e714c348fb510c5fde0c887 P 602c82eac0e346bfbf403636010c9077: No bootstrap required, opened a new log
I20260811 05:15:33.602849 21031 ts_tablet_manager.cc:1397] T 8a4647064e714c348fb510c5fde0c887 P 602c82eac0e346bfbf403636010c9077: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260811 05:15:33.602903 21035 ts_tablet_manager.cc:1397] T 8a4647064e714c348fb510c5fde0c887 P a611f645ffae454a8ec3292f0e20d1b7: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 05:15:33.605605 21035 raft_consensus.cc:348] T 8a4647064e714c348fb510c5fde0c887 P a611f645ffae454a8ec3292f0e20d1b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c4f1facc964e7bb58e28804425d1d2" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 43961 } } peers { permanent_uuid: "a611f645ffae454a8ec3292f0e20d1b7" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 39447 } } peers { permanent_uuid: "602c82eac0e346bfbf403636010c9077" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 45993 } }
I20260811 05:15:33.605826 21035 raft_consensus.cc:374] T 8a4647064e714c348fb510c5fde0c887 P a611f645ffae454a8ec3292f0e20d1b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:15:33.605859 21035 raft_consensus.cc:729] T 8a4647064e714c348fb510c5fde0c887 P a611f645ffae454a8ec3292f0e20d1b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a611f645ffae454a8ec3292f0e20d1b7, State: Initialized, Role: FOLLOWER
I20260811 05:15:33.605726 21031 raft_consensus.cc:348] T 8a4647064e714c348fb510c5fde0c887 P 602c82eac0e346bfbf403636010c9077 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c4f1facc964e7bb58e28804425d1d2" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 43961 } } peers { permanent_uuid: "a611f645ffae454a8ec3292f0e20d1b7" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 39447 } } peers { permanent_uuid: "602c82eac0e346bfbf403636010c9077" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 45993 } }
I20260811 05:15:33.605922 21031 raft_consensus.cc:374] T 8a4647064e714c348fb510c5fde0c887 P 602c82eac0e346bfbf403636010c9077 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:15:33.605962 21031 raft_consensus.cc:729] T 8a4647064e714c348fb510c5fde0c887 P 602c82eac0e346bfbf403636010c9077 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602c82eac0e346bfbf403636010c9077, State: Initialized, Role: FOLLOWER
I20260811 05:15:33.606034 21035 consensus_queue.cc:260] T 8a4647064e714c348fb510c5fde0c887 P a611f645ffae454a8ec3292f0e20d1b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c4f1facc964e7bb58e28804425d1d2" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 43961 } } peers { permanent_uuid: "a611f645ffae454a8ec3292f0e20d1b7" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 39447 } } peers { permanent_uuid: "602c82eac0e346bfbf403636010c9077" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 45993 } }
I20260811 05:15:33.606101 21031 consensus_queue.cc:260] T 8a4647064e714c348fb510c5fde0c887 P 602c82eac0e346bfbf403636010c9077 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c4f1facc964e7bb58e28804425d1d2" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 43961 } } peers { permanent_uuid: "a611f645ffae454a8ec3292f0e20d1b7" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 39447 } } peers { permanent_uuid: "602c82eac0e346bfbf403636010c9077" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 45993 } }
I20260811 05:15:33.606424 21035 ts_tablet_manager.cc:1428] T 8a4647064e714c348fb510c5fde0c887 P a611f645ffae454a8ec3292f0e20d1b7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 05:15:33.606578 20502 heartbeater.cc:503] Master 127.28.143.190:34353 was elected leader, sending a full tablet report...
I20260811 05:15:33.606753 21031 ts_tablet_manager.cc:1428] T 8a4647064e714c348fb510c5fde0c887 P 602c82eac0e346bfbf403636010c9077: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260811 05:15:33.607164 20726 heartbeater.cc:503] Master 127.28.143.190:34353 was elected leader, sending a full tablet report...
W20260811 05:15:33.610100 20503 tablet.cc:2369] T 8a4647064e714c348fb510c5fde0c887 P a611f645ffae454a8ec3292f0e20d1b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:15:33.641379 21040 raft_consensus.cc:482] T 8a4647064e714c348fb510c5fde0c887 P a611f645ffae454a8ec3292f0e20d1b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:15:33.641606 21040 raft_consensus.cc:504] T 8a4647064e714c348fb510c5fde0c887 P a611f645ffae454a8ec3292f0e20d1b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c4f1facc964e7bb58e28804425d1d2" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 43961 } } peers { permanent_uuid: "a611f645ffae454a8ec3292f0e20d1b7" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 39447 } } peers { permanent_uuid: "602c82eac0e346bfbf403636010c9077" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 45993 } }
I20260811 05:15:33.642220 21040 leader_election.cc:288] T 8a4647064e714c348fb510c5fde0c887 P a611f645ffae454a8ec3292f0e20d1b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3c4f1facc964e7bb58e28804425d1d2 (127.28.143.131:43961), 602c82eac0e346bfbf403636010c9077 (127.28.143.130:45993)
I20260811 05:15:33.646755 21024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:33.642763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:59547 (local address 127.28.143.131:43961)
0811 05:15:33.642808 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:33.642810 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:33.642902 (+ 92us) server_negotiation.cc:407] Connection header received
0811 05:15:33.643016 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:33.643018 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:33.643041 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:33.643073 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:15:33.643387 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:33.644132 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:33.645626 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:33.646202 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:33.646278 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:33.646288 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:33.646645 (+ 357us) server_negotiation.cc:299] Negotiation successful
0811 05:15:33.646683 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 05:15:33.646922 21043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:33.642431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.131:43961 (local address 127.28.143.129:59547)
0811 05:15:33.642788 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:33.642803 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:33.642938 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:33.643152 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:33.643153 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:33.643155 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:15:33.643244 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:33.643247 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:33.644243 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:33.644244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:33.645512 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:33.645516 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:33.646430 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:33.646432 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:33.646555 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:33.646562 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 05:15:33.646635 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 05:15:33.646681 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":193,"threads_started":1}
I20260811 05:15:33.647352 20880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a4647064e714c348fb510c5fde0c887" candidate_uuid: "a611f645ffae454a8ec3292f0e20d1b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3c4f1facc964e7bb58e28804425d1d2" is_pre_election: true
I20260811 05:15:33.647524 20880 raft_consensus.cc:2454] T 8a4647064e714c348fb510c5fde0c887 P e3c4f1facc964e7bb58e28804425d1d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a611f645ffae454a8ec3292f0e20d1b7 in term 0.
I20260811 05:15:33.647850 20298 leader_election.cc:302] T 8a4647064e714c348fb510c5fde0c887 P a611f645ffae454a8ec3292f0e20d1b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a611f645ffae454a8ec3292f0e20d1b7, e3c4f1facc964e7bb58e28804425d1d2; no voters:
I20260811 05:15:33.648044 21040 raft_consensus.cc:2781] T 8a4647064e714c348fb510c5fde0c887 P a611f645ffae454a8ec3292f0e20d1b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:15:33.648097 21040 raft_consensus.cc:482] T 8a4647064e714c348fb510c5fde0c887 P a611f645ffae454a8ec3292f0e20d1b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:15:33.648120 21040 raft_consensus.cc:3037] T 8a4647064e714c348fb510c5fde0c887 P a611f645ffae454a8ec3292f0e20d1b7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:15:33.649070 21040 raft_consensus.cc:504] T 8a4647064e714c348fb510c5fde0c887 P a611f645ffae454a8ec3292f0e20d1b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c4f1facc964e7bb58e28804425d1d2" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 43961 } } peers { permanent_uuid: "a611f645ffae454a8ec3292f0e20d1b7" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 39447 } } peers { permanent_uuid: "602c82eac0e346bfbf403636010c9077" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 45993 } }
I20260811 05:15:33.649467 20880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a4647064e714c348fb510c5fde0c887" candidate_uuid: "a611f645ffae454a8ec3292f0e20d1b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3c4f1facc964e7bb58e28804425d1d2"
I20260811 05:15:33.649549 20880 raft_consensus.cc:3037] T 8a4647064e714c348fb510c5fde0c887 P e3c4f1facc964e7bb58e28804425d1d2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:15:33.650660 21030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:33.643298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:58361 (local address 127.28.143.130:45993)
0811 05:15:33.643359 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:33.643361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:33.647243 (+ 3882us) server_negotiation.cc:407] Connection header received
0811 05:15:33.647482 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:33.647483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:33.647495 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:33.647523 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:15:33.647694 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:33.648406 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:33.649712 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:33.650289 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:33.650344 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:33.650353 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:33.650546 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 05:15:33.650590 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 05:15:33.650702 20880 raft_consensus.cc:2454] T 8a4647064e714c348fb510c5fde0c887 P e3c4f1facc964e7bb58e28804425d1d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a611f645ffae454a8ec3292f0e20d1b7 in term 1.
I20260811 05:15:33.650887 21043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:33.642972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:45993 (local address 127.28.143.129:58361)
0811 05:15:33.647182 (+ 4210us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:33.647191 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:33.647381 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:33.647590 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:33.647591 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:33.647592 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:15:33.647622 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:33.647624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:33.648512 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:33.648513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:33.649602 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:33.649605 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:33.650412 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:33.650412 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:33.650473 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:33.650476 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 05:15:33.650774 (+ 298us) client_negotiation.cc:240] Negotiation successful
0811 05:15:33.650819 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4197,"thread_start_us":217,"threads_started":1}
I20260811 05:15:33.651183 20298 leader_election.cc:302] T 8a4647064e714c348fb510c5fde0c887 P a611f645ffae454a8ec3292f0e20d1b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a611f645ffae454a8ec3292f0e20d1b7, e3c4f1facc964e7bb58e28804425d1d2; no voters:
I20260811 05:15:33.651378 20653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a4647064e714c348fb510c5fde0c887" candidate_uuid: "a611f645ffae454a8ec3292f0e20d1b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602c82eac0e346bfbf403636010c9077" is_pre_election: true
I20260811 05:15:33.651531 21045 raft_consensus.cc:2781] T 8a4647064e714c348fb510c5fde0c887 P a611f645ffae454a8ec3292f0e20d1b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:15:33.651607 20653 raft_consensus.cc:2454] T 8a4647064e714c348fb510c5fde0c887 P 602c82eac0e346bfbf403636010c9077 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a611f645ffae454a8ec3292f0e20d1b7 in term 0.
I20260811 05:15:33.651747 21045 raft_consensus.cc:686] T 8a4647064e714c348fb510c5fde0c887 P a611f645ffae454a8ec3292f0e20d1b7 [term 1 LEADER]: Becoming Leader. State: Replica: a611f645ffae454a8ec3292f0e20d1b7, State: Running, Role: LEADER
I20260811 05:15:33.651739 20652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a4647064e714c348fb510c5fde0c887" candidate_uuid: "a611f645ffae454a8ec3292f0e20d1b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602c82eac0e346bfbf403636010c9077"
I20260811 05:15:33.651895 21045 consensus_queue.cc:237] T 8a4647064e714c348fb510c5fde0c887 P a611f645ffae454a8ec3292f0e20d1b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c4f1facc964e7bb58e28804425d1d2" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 43961 } } peers { permanent_uuid: "a611f645ffae454a8ec3292f0e20d1b7" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 39447 } } peers { permanent_uuid: "602c82eac0e346bfbf403636010c9077" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 45993 } }
I20260811 05:15:33.651927 20652 raft_consensus.cc:3037] T 8a4647064e714c348fb510c5fde0c887 P 602c82eac0e346bfbf403636010c9077 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:15:33.652890 20132 catalog_manager.cc:5495] T 8a4647064e714c348fb510c5fde0c887 P a611f645ffae454a8ec3292f0e20d1b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a611f645ffae454a8ec3292f0e20d1b7 (127.28.143.129). New cstate: current_term: 1 leader_uuid: "a611f645ffae454a8ec3292f0e20d1b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3c4f1facc964e7bb58e28804425d1d2" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 43961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a611f645ffae454a8ec3292f0e20d1b7" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 39447 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "602c82eac0e346bfbf403636010c9077" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 45993 } health_report { overall_health: UNKNOWN } } }
I20260811 05:15:33.653602 20652 raft_consensus.cc:2454] T 8a4647064e714c348fb510c5fde0c887 P 602c82eac0e346bfbf403636010c9077 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a611f645ffae454a8ec3292f0e20d1b7 in term 1.
I20260811 05:15:33.655659 21040 leader_election.cc:288] T 8a4647064e714c348fb510c5fde0c887 P a611f645ffae454a8ec3292f0e20d1b7 [CANDIDATE]: Term 1 election: Requested vote from peers e3c4f1facc964e7bb58e28804425d1d2 (127.28.143.131:43961), 602c82eac0e346bfbf403636010c9077 (127.28.143.130:45993)
I20260811 05:15:33.661507 21026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:33.658102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38776 (local address 127.28.143.129:39447)
0811 05:15:33.658141 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:33.658143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:33.658154 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:15:33.658193 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:33.658194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:33.658214 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:33.658303 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:15:33.658437 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:33.659178 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:33.660312 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:33.660666 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:33.660724 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:33.660932 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:15:33.661084 (+ 152us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:15:33.661155 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:33.661402 (+ 247us) server_negotiation.cc:299] Negotiation successful
0811 05:15:33.661426 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260811 05:15:33.663895 20652 raft_consensus.cc:1261] T 8a4647064e714c348fb510c5fde0c887 P 602c82eac0e346bfbf403636010c9077 [term 1 FOLLOWER]: Refusing update from remote peer a611f645ffae454a8ec3292f0e20d1b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:15:33.664052 20880 raft_consensus.cc:1261] T 8a4647064e714c348fb510c5fde0c887 P e3c4f1facc964e7bb58e28804425d1d2 [term 1 FOLLOWER]: Refusing update from remote peer a611f645ffae454a8ec3292f0e20d1b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:15:33.664397 21040 consensus_queue.cc:1035] T 8a4647064e714c348fb510c5fde0c887 P a611f645ffae454a8ec3292f0e20d1b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "602c82eac0e346bfbf403636010c9077" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 45993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:15:33.664710 21048 consensus_queue.cc:1035] T 8a4647064e714c348fb510c5fde0c887 P a611f645ffae454a8ec3292f0e20d1b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3c4f1facc964e7bb58e28804425d1d2" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 43961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:15:33.668061 21053 mvcc.cc:205] Tried to move back new op lower bound from 7317198166684254208 to 7317198166639075328. Current Snapshot: MvccSnapshot[applied={T|T < 7317198166684254208}]
I20260811 05:15:33.668174 21054 mvcc.cc:205] Tried to move back new op lower bound from 7317198166684254208 to 7317198166639075328. Current Snapshot: MvccSnapshot[applied={T|T < 7317198166684254208}]
I20260811 05:15:33.670349 21055 mvcc.cc:205] Tried to move back new op lower bound from 7317198166684254208 to 7317198166639075328. Current Snapshot: MvccSnapshot[applied={T|T < 7317198166684254208}]
I20260811 05:15:33.672644 20132 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46500:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:15:33.672760 20132 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46500:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:15:33.673542 20132 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fe432f74cc3e45e0a8d562b4653559e8: Sending DeleteTablet for 3 replicas of tablet 8a4647064e714c348fb510c5fde0c887
I20260811 05:15:33.673913 20851 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a4647064e714c348fb510c5fde0c887 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:15:33 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:58890
I20260811 05:15:33.674001 20617 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a4647064e714c348fb510c5fde0c887 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:15:33 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:56510
I20260811 05:15:33.674175 20378 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a4647064e714c348fb510c5fde0c887 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:15:33 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:38770
I20260811 05:15:33.674240 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 20201
I20260811 05:15:33.674290 21069 tablet_replica.cc:330] T 8a4647064e714c348fb510c5fde0c887 P e3c4f1facc964e7bb58e28804425d1d2: stopping tablet replica
I20260811 05:15:33.674400 21069 raft_consensus.cc:2229] T 8a4647064e714c348fb510c5fde0c887 P e3c4f1facc964e7bb58e28804425d1d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:15:33.674575 21069 raft_consensus.cc:2258] T 8a4647064e714c348fb510c5fde0c887 P e3c4f1facc964e7bb58e28804425d1d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:15:33.693550 21069 ts_tablet_manager.cc:1905] T 8a4647064e714c348fb510c5fde0c887 P e3c4f1facc964e7bb58e28804425d1d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:15:33.694200 21071 tablet_replica.cc:330] T 8a4647064e714c348fb510c5fde0c887 P 602c82eac0e346bfbf403636010c9077: stopping tablet replica
I20260811 05:15:33.694398 21071 raft_consensus.cc:2229] T 8a4647064e714c348fb510c5fde0c887 P 602c82eac0e346bfbf403636010c9077 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:15:33.694588 21071 raft_consensus.cc:2258] T 8a4647064e714c348fb510c5fde0c887 P 602c82eac0e346bfbf403636010c9077 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:15:33.695142 21071 ts_tablet_manager.cc:1905] T 8a4647064e714c348fb510c5fde0c887 P 602c82eac0e346bfbf403636010c9077: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:15:33.696141 21069 ts_tablet_manager.cc:1918] T 8a4647064e714c348fb510c5fde0c887 P e3c4f1facc964e7bb58e28804425d1d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:15:33.696291 21069 log.cc:1196] T 8a4647064e714c348fb510c5fde0c887 P e3c4f1facc964e7bb58e28804425d1d2: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-2/wal/wals/8a4647064e714c348fb510c5fde0c887
I20260811 05:15:33.696719 21069 ts_tablet_manager.cc:1939] T 8a4647064e714c348fb510c5fde0c887 P e3c4f1facc964e7bb58e28804425d1d2: Deleting consensus metadata
I20260811 05:15:33.696995 21071 ts_tablet_manager.cc:1918] T 8a4647064e714c348fb510c5fde0c887 P 602c82eac0e346bfbf403636010c9077: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:15:33.697125 21071 log.cc:1196] T 8a4647064e714c348fb510c5fde0c887 P 602c82eac0e346bfbf403636010c9077: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425159060450-29246-0/minicluster-data/ts-1/wal/wals/8a4647064e714c348fb510c5fde0c887
I20260811 05:15:33.697408 21071 ts_tablet_manager.cc:1939] T 8a4647064e714c348fb510c5fde0c887 P 602c82eac0e346bfbf403636010c9077: Deleting consensus metadata
I20260811 05:15:33.697445 20103 catalog_manager.cc:4841] TS e3c4f1facc964e7bb58e28804425d1d2 (127.28.143.131:43961): tablet 8a4647064e714c348fb510c5fde0c887 (table test-table [id=940ede0223a440f88229e922acf5ad90]) successfully deleted
I20260811 05:15:33.697881 20108 catalog_manager.cc:4841] TS 602c82eac0e346bfbf403636010c9077 (127.28.143.130:45993): tablet 8a4647064e714c348fb510c5fde0c887 (table test-table [id=940ede0223a440f88229e922acf5ad90]) successfully deleted
W20260811 05:15:33.698889 20106 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.143.129:39447 (error 108)
I20260811 05:15:33.699466 21025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:33.699330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:39447 (local address 127.0.0.1:38784)
0811 05:15:33.699388 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:33.699427 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.129:39447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 05:15:33.699600 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 20509
W20260811 05:15:33.699716 20106 catalog_manager.cc:4584] TS a611f645ffae454a8ec3292f0e20d1b7 (127.28.143.129:39447): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8a4647064e714c348fb510c5fde0c887: Network error: Client connection negotiation failed: client connection to 127.28.143.129:39447: connect: Connection refused (error 111)
I20260811 05:15:33.733155 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 20730
I20260811 05:15:33.744454 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 20073
I20260811 05:15:33.772456 29246 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 05:15:34.118000 29246 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 05:15:34.118106 29246 mini_ranger.cc:66] Stopping Ranger...
I20260811 05:15:34.473618 29246 mini_ranger.cc:68] Stopped Ranger
2026-08-11 05:15:34.473 UTC [18400] LOG: received smart shutdown request
2026-08-11 05:15:34.479 UTC [18400] LOG: background worker "logical replication launcher" (PID 18419) exited with exit code 1
2026-08-11 05:15:34.479 UTC [18405] LOG: shutting down
2026-08-11 05:15:34.678 UTC [18400] LOG: database system is shut down
2026-08-11T05:15:34Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (53879 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:15:34 dist-test-slave-f7th krb5kdc[21092](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:15:34 dist-test-slave-f7th krb5kdc[21092](info): set up 2 sockets
Aug 11 05:15:34 dist-test-slave-f7th krb5kdc[21092](info): commencing operation
krb5kdc: starting...
W20260811 05:15:35.991137 29246 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Aug 11 05:15:36 dist-test-slave-f7th krb5kdc[21092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:15:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:15:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
I20260811 05:15:36.124109 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:34367
--webserver_interface=127.28.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:37807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:34367,127.28.143.189:36007,127.28.143.188:35905
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:15:36.210126 21110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:15:36.210366 21110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:15:36.210391 21110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:15:36.211611 21110 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:15:36.211659 21110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:15:36.211673 21110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:15:36.211684 21110 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:15:36.211695 21110 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:15:36.213405 21110 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:37807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:34367,127.28.143.189:36007,127.28.143.188:35905
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:34367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:15:36.213660 21110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:15:36.213858 21110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:15:36.219059 21129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:36.219226 21125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:36.219098 21127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:36.219359 21110 server_base.cc:1034] running on GCE node
I20260811 05:15:36.219657 21110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:15:36.220019 21110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:15:36.221185 21110 hybrid_clock.cc:648] HybridClock initialized: now 1786425336221166 us; error 47 us; skew 500 ppm
Aug 11 05:15:36 dist-test-slave-f7th krb5kdc[21092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425336, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:15:36.227499 21110 init.cc:377] Logged in from keytab as kudu/127.28.143.190@KRBTEST.COM (short username kudu)
I20260811 05:15:36.228029 21110 webserver.cc:460] Webserver started at http://127.28.143.190:37985/ using document root <none> and password file <none>
I20260811 05:15:36.228272 21110 fs_manager.cc:359] Metadata directory not provided
I20260811 05:15:36.228329 21110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:15:36.228482 21110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:15:36.229753 21110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "6169480bab2a4508b836503948b135db"
format_stamp: "Formatted at 2026-08-11 05:15:36 on dist-test-slave-f7th"
server_key: "ba75c5de2e789c5253c9c1d26a5fb263"
server_key_iv: "63f52a5dfefc6dfc0ac2516dbb0d42bc"
server_key_version: "encryptionkey@0"
I20260811 05:15:36.230149 21110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "6169480bab2a4508b836503948b135db"
format_stamp: "Formatted at 2026-08-11 05:15:36 on dist-test-slave-f7th"
server_key: "ba75c5de2e789c5253c9c1d26a5fb263"
server_key_iv: "63f52a5dfefc6dfc0ac2516dbb0d42bc"
server_key_version: "encryptionkey@0"
I20260811 05:15:36.231688 21110 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 05:15:36.232743 21140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:36.232983 21110 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:15:36.233091 21110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "6169480bab2a4508b836503948b135db"
format_stamp: "Formatted at 2026-08-11 05:15:36 on dist-test-slave-f7th"
server_key: "ba75c5de2e789c5253c9c1d26a5fb263"
server_key_iv: "63f52a5dfefc6dfc0ac2516dbb0d42bc"
server_key_version: "encryptionkey@0"
I20260811 05:15:36.233177 21110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:15:36.241760 21110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:15:36.244099 21110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:15:36.244268 21110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:15:36.248658 21110 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:34367
I20260811 05:15:36.248762 21248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:34367 every 8 connection(s)
I20260811 05:15:36.249114 21110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:15:36.250093 21250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:15:36.250547 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 21110
I20260811 05:15:36.250649 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:15:36.250828 29246 external_mini_cluster.cc:1428] Setting key 905feff40452b67879e3ebf840759849
I20260811 05:15:36.251824 21250 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.143.190" port: 34367 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 05:15:36 dist-test-slave-f7th krb5kdc[21092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425336, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.28.143.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.189@KRBTEST.COM" created.
I20260811 05:15:36.262657 21254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:36.252756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:34367 (local address 127.0.0.1:53758)
0811 05:15:36.252958 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:36.253010 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:36.253638 (+ 628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:36.254094 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:36.254097 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:36.254477 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:15:36.254725 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:36.254731 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.255864 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.255865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:36.256648 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:36.256652 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.257154 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.257156 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:36.257224 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:36.257773 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:15:36.257783 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:15:36.259259 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:15:36.261117 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:36.261127 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:36.261130 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:36.261362 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:36.261608 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:36.261609 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:36.261610 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:36.261699 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:36.262060 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:15:36.262065 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:15:36.262132 (+ 67us) client_negotiation.cc:769] Sending connection context
0811 05:15:36.262217 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 05:15:36.262465 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":93,"thread_start_us":102,"threads_started":1}
I20260811 05:15:36.262660 21255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:36.252795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53758 (local address 127.28.143.190:34367)
0811 05:15:36.253836 (+ 1041us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:36.253840 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:36.253859 (+ 19us) server_negotiation.cc:407] Connection header received
0811 05:15:36.253939 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:36.253941 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:36.253958 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:36.254036 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:36.254874 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.255744 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.256732 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.257070 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.257177 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:36.259347 (+ 2170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:36.259356 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:36.259356 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:36.259364 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:36.260973 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:36.261454 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:36.261458 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:36.261459 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:36.261546 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:36.261755 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:36.261756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:36.261756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:36.261993 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:36.262072 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:36.262252 (+ 180us) server_negotiation.cc:299] Negotiation successful
0811 05:15:36.262397 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1012,"thread_start_us":81,"threads_started":1}
I20260811 05:15:36.263489 21250 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.143.189" port: 36007 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:15:36.263926 21254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:36.263771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.189:36007 (local address 127.0.0.1:51690)
0811 05:15:36.263835 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:36.263896 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.189:36007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 05:15:36.264096 21144 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.143.189:36007: connect: Connection refused (error 111)
I20260811 05:15:36.264642 21254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:36.264496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.189:36007 (local address 127.0.0.1:51704)
0811 05:15:36.264562 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:36.264590 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.189:36007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 05:15:36.264741 21250 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.143.189:36007: Network error: Client connection negotiation failed: client connection to 127.28.143.189:36007: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.189.keytab.
Entry for principal kudu/127.28.143.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.189.keytab.
I20260811 05:15:36.280081 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.189:36007
--webserver_interface=127.28.143.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.189.keytab
--principal=kudu/127.28.143.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:37807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:34367,127.28.143.189:36007,127.28.143.188:35905
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:15:36.314103 21250 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.143.189" port: 36007 } attempt: 1
I20260811 05:15:36.314698 21254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:36.314557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.189:36007 (local address 127.0.0.1:51708)
0811 05:15:36.314624 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:36.314656 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.189:36007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 05:15:36.317060 21254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:36.316914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.189:36007 (local address 127.0.0.1:51724)
0811 05:15:36.316975 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:36.317021 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.189:36007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260811 05:15:36.317206 21250 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.143.189:36007: Network error: Client connection negotiation failed: client connection to 127.28.143.189:36007: connect: Connection refused (error 111)
W20260811 05:15:36.374972 21259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:15:36.375254 21259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:15:36.375291 21259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:15:36.376506 21259 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:15:36.376552 21259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:15:36.376571 21259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:15:36.376588 21259 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:15:36.376603 21259 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:15:36.378278 21259 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:37807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:34367,127.28.143.189:36007,127.28.143.188:35905
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.189.keytab
--principal=kudu/127.28.143.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.189:36007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.28.143.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21259
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:15:36.378547 21259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:15:36.378764 21259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:15:36.383527 21270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:36.383518 21272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:36.383553 21276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:36.383769 21259 server_base.cc:1034] running on GCE node
I20260811 05:15:36.384080 21259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:15:36.384379 21259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:15:36.385363 21250 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.143.189" port: 36007 } attempt: 2
I20260811 05:15:36.385562 21259 hybrid_clock.cc:648] HybridClock initialized: now 1786425336385551 us; error 38 us; skew 500 ppm
I20260811 05:15:36.385773 21254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:36.385655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.189:36007 (local address 127.0.0.1:51734)
0811 05:15:36.385709 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:36.385739 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.189:36007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260811 05:15:36.386065 21254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:36.385983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.189:36007 (local address 127.0.0.1:51746)
0811 05:15:36.386019 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:36.386043 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.189:36007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260811 05:15:36.386147 21250 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.143.189:36007: Network error: Client connection negotiation failed: client connection to 127.28.143.189:36007: connect: Connection refused (error 111)
Aug 11 05:15:36 dist-test-slave-f7th krb5kdc[21092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425336, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:15:36.388489 21259 init.cc:377] Logged in from keytab as kudu/127.28.143.189@KRBTEST.COM (short username kudu)
I20260811 05:15:36.389156 21259 webserver.cc:460] Webserver started at http://127.28.143.189:45829/ using document root <none> and password file <none>
I20260811 05:15:36.389437 21259 fs_manager.cc:359] Metadata directory not provided
I20260811 05:15:36.389495 21259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:15:36.389635 21259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:15:36.390791 21259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-1/data/instance:
uuid: "3845ec660a6f4c6d8185b2cabb589a37"
format_stamp: "Formatted at 2026-08-11 05:15:36 on dist-test-slave-f7th"
server_key: "fb561622dfce81ef087b935d934dca80"
server_key_iv: "2605088e1ca3b2b5d2afb1c114b49750"
server_key_version: "encryptionkey@0"
I20260811 05:15:36.391146 21259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-1/wal/instance:
uuid: "3845ec660a6f4c6d8185b2cabb589a37"
format_stamp: "Formatted at 2026-08-11 05:15:36 on dist-test-slave-f7th"
server_key: "fb561622dfce81ef087b935d934dca80"
server_key_iv: "2605088e1ca3b2b5d2afb1c114b49750"
server_key_version: "encryptionkey@0"
I20260811 05:15:36.392587 21259 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 05:15:36.393560 21284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:36.393802 21259 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:15:36.393909 21259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-1/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-1/wal
uuid: "3845ec660a6f4c6d8185b2cabb589a37"
format_stamp: "Formatted at 2026-08-11 05:15:36 on dist-test-slave-f7th"
server_key: "fb561622dfce81ef087b935d934dca80"
server_key_iv: "2605088e1ca3b2b5d2afb1c114b49750"
server_key_version: "encryptionkey@0"
I20260811 05:15:36.394008 21259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:15:36.414461 21259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:15:36.417086 21259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:15:36.417256 21259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:15:36.421216 21259 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.189:36007
I20260811 05:15:36.421252 21383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.189:36007 every 8 connection(s)
I20260811 05:15:36.421706 21259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-1/data/info.pb
I20260811 05:15:36.422650 21384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:15:36.424256 21384 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.143.190" port: 34367 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:15:36.428200 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 21259
I20260811 05:15:36.428300 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-1/wal/instance
I20260811 05:15:36.428472 29246 external_mini_cluster.cc:1428] Setting key d17c3c08f5e4abc52251b977b967e0aa
Aug 11 05:15:36 dist-test-slave-f7th krb5kdc[21092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425336, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.189@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:15:36.434226 21255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:36.425091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53762 (local address 127.28.143.190:34367)
0811 05:15:36.425149 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:36.425151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:36.425359 (+ 208us) server_negotiation.cc:407] Connection header received
0811 05:15:36.425873 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:36.425875 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:36.425889 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:36.425949 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:36.426604 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.427146 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.428118 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.428347 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.428403 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:36.431004 (+ 2601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:36.431006 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:36.431006 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:36.431017 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:36.432524 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:36.433100 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:36.433100 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:36.433101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:36.433158 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:36.433474 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:36.433476 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:36.433476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:36.433615 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:36.433681 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:36.434069 (+ 388us) server_negotiation.cc:299] Negotiation successful
0811 05:15:36.434156 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 05:15:36.434500 21390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:36.425029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:34367 (local address 127.0.0.1:53762)
0811 05:15:36.425233 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:36.425272 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:36.425774 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:36.426026 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:36.426030 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:36.426279 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:15:36.426515 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:36.426519 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.427237 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.427239 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:36.428040 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:36.428042 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.428441 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.428443 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:36.428679 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:36.429219 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:15:36.429228 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:15:36.430900 (+ 1672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:15:36.432645 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:36.432654 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:36.432657 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:36.433024 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:36.433221 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:36.433222 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:36.433223 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:36.433384 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:36.433687 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:15:36.433695 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:15:36.433889 (+ 194us) client_negotiation.cc:769] Sending connection context
0811 05:15:36.434015 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 05:15:36.434228 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":102,"threads_started":1}
I20260811 05:15:36.435138 21384 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.143.189" port: 36007 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.28.143.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.188@KRBTEST.COM" created.
Aug 11 05:15:36 dist-test-slave-f7th krb5kdc[21092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425336, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.189@KRBTEST.COM for kudu/127.28.143.189@KRBTEST.COM
I20260811 05:15:36.445815 21390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:36.435409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.189:36007 (local address 127.0.0.1:51748)
0811 05:15:36.435470 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:36.435485 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:36.435535 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:36.435781 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:36.435783 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:36.435956 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:15:36.436023 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:36.436025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.437034 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.437036 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:36.437816 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:36.437819 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.438449 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.438450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:36.438520 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:36.438986 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:15:36.438992 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:15:36.442226 (+ 3234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:15:36.444339 (+ 2113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:36.444340 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:36.444341 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:36.444545 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:36.444911 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:36.444912 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:36.444912 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:36.444990 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:36.445432 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:15:36.445433 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:15:36.445497 (+ 64us) client_negotiation.cc:769] Sending connection context
0811 05:15:36.445582 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 05:15:36.445709 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260811 05:15:36.445930 21392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:36.435472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51748 (local address 127.28.143.189:36007)
0811 05:15:36.435645 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:36.435650 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:36.435668 (+ 18us) server_negotiation.cc:407] Connection header received
0811 05:15:36.435689 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:36.435691 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:36.435702 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:36.435766 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:36.436104 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.436948 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.438030 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.438350 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.438415 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:36.442355 (+ 3940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:36.442369 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:36.442369 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:36.442378 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:36.444224 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:36.444679 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:36.444683 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:36.444683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:36.444782 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:36.445066 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:36.445067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:36.445067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:36.445317 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:36.445404 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:36.445628 (+ 224us) server_negotiation.cc:299] Negotiation successful
0811 05:15:36.445804 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":141,"thread_start_us":93,"threads_started":1}
I20260811 05:15:36.447926 21384 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.143.188" port: 35905 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:15:36.448443 21390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:36.448272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:35905 (local address 127.0.0.1:43424)
0811 05:15:36.448326 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:36.448405 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.188:35905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260811 05:15:36.448797 21294 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.143.188:35905: connect: Connection refused (error 111)
I20260811 05:15:36.449363 21390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:36.449254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:35905 (local address 127.0.0.1:43436)
0811 05:15:36.449305 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:36.449338 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.188:35905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260811 05:15:36.449488 21384 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.143.188:35905: Network error: Client connection negotiation failed: client connection to 127.28.143.188:35905: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.188.keytab.
Entry for principal kudu/127.28.143.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.188.keytab.
I20260811 05:15:36.465149 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.188:35905
--webserver_interface=127.28.143.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.188.keytab
--principal=kudu/127.28.143.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:37807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:34367,127.28.143.189:36007,127.28.143.188:35905
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:15:36.477284 21250 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.143.189" port: 36007 } attempt: 3
Aug 11 05:15:36 dist-test-slave-f7th krb5kdc[21092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425336, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for kudu/127.28.143.189@KRBTEST.COM
I20260811 05:15:36.485291 21392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:36.477803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51764 (local address 127.28.143.189:36007)
0811 05:15:36.477861 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:36.477863 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:36.477874 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:15:36.477950 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:36.477951 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:36.477962 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:36.478019 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:36.478434 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.478959 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.480014 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.480245 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.480318 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:36.482441 (+ 2123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:36.482443 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:36.482443 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:36.482451 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:36.484023 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:36.484484 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:36.484485 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:36.484486 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:36.484549 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:36.484788 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:36.484788 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:36.484788 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:36.484937 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:36.484990 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:36.485144 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 05:15:36.485218 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 05:15:36.485338 21254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:36.477722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.189:36007 (local address 127.0.0.1:51764)
0811 05:15:36.477806 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:36.477821 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:36.477886 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:36.478034 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:36.478035 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:36.478237 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:15:36.478314 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:36.478316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.479040 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.479040 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:36.479910 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:36.479913 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.480325 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.480325 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:36.480402 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:36.480901 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:15:36.480907 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:15:36.482333 (+ 1426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:15:36.484136 (+ 1803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:36.484138 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:36.484138 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:36.484381 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:36.484646 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:36.484646 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:36.484647 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:36.484718 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:36.484997 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:15:36.484998 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:15:36.485055 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 05:15:36.485124 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 05:15:36.485233 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 05:15:36.485836 21250 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.143.188" port: 35905 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:15:36.486214 21254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:36.486106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:35905 (local address 127.0.0.1:43452)
0811 05:15:36.486150 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:36.486188 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.188:35905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260811 05:15:36.486577 21254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:36.486494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:35905 (local address 127.0.0.1:43466)
0811 05:15:36.486529 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:36.486557 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.188:35905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260811 05:15:36.486667 21250 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.143.188:35905: Network error: Client connection negotiation failed: client connection to 127.28.143.188:35905: connect: Connection refused (error 111)
I20260811 05:15:36.498737 21384 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.143.188" port: 35905 } attempt: 1
I20260811 05:15:36.499269 21390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:36.499105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:35905 (local address 127.0.0.1:43470)
0811 05:15:36.499172 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:36.499226 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.188:35905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260811 05:15:36.499648 21390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:36.499548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:35905 (local address 127.0.0.1:43476)
0811 05:15:36.499595 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:36.499626 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.188:35905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260811 05:15:36.499749 21384 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.143.188:35905: Network error: Client connection negotiation failed: client connection to 127.28.143.188:35905: connect: Connection refused (error 111)
I20260811 05:15:36.517853 21250 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.143.188" port: 35905 } attempt: 1
I20260811 05:15:36.518472 21254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:36.518328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:35905 (local address 127.0.0.1:43488)
0811 05:15:36.518397 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:36.518443 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.188:35905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 05:15:36.518913 21254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:36.518738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:35905 (local address 127.0.0.1:43500)
0811 05:15:36.518809 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:36.518885 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.188:35905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 05:15:36.519011 21250 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.143.188:35905: Network error: Client connection negotiation failed: client connection to 127.28.143.188:35905: connect: Connection refused (error 111)
W20260811 05:15:36.554538 21398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:15:36.554774 21398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:15:36.554798 21398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:15:36.555994 21398 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:15:36.556067 21398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:15:36.556083 21398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:15:36.556095 21398 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:15:36.556108 21398 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:15:36.557792 21398 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:37807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:34367,127.28.143.189:36007,127.28.143.188:35905
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.188.keytab
--principal=kudu/127.28.143.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.188:35905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.28.143.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:15:36.558080 21398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:15:36.558308 21398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:15:36.563374 21418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:36.563477 21398 server_base.cc:1034] running on GCE node
W20260811 05:15:36.563374 21412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:36.563438 21411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:36.563906 21398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:15:36.564234 21398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:15:36.565376 21398 hybrid_clock.cc:648] HybridClock initialized: now 1786425336565362 us; error 29 us; skew 500 ppm
Aug 11 05:15:36 dist-test-slave-f7th krb5kdc[21092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425336, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:15:36.567904 21384 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.143.188" port: 35905 } attempt: 2
I20260811 05:15:36.568086 21398 init.cc:377] Logged in from keytab as kudu/127.28.143.188@KRBTEST.COM (short username kudu)
I20260811 05:15:36.568349 21390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:36.568220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:35905 (local address 127.0.0.1:43504)
0811 05:15:36.568272 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:36.568309 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.188:35905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260811 05:15:36.568701 21398 webserver.cc:460] Webserver started at http://127.28.143.188:36163/ using document root <none> and password file <none>
I20260811 05:15:36.568714 21390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:36.568637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:35905 (local address 127.0.0.1:43506)
0811 05:15:36.568670 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:36.568696 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.188:35905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260811 05:15:36.568954 21384 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.143.188:35905: Network error: Client connection negotiation failed: client connection to 127.28.143.188:35905: connect: Connection refused (error 111)
I20260811 05:15:36.569195 21398 fs_manager.cc:359] Metadata directory not provided
I20260811 05:15:36.569264 21398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:15:36.569474 21398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:15:36.570735 21398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-2/data/instance:
uuid: "b6e28cd99ac8431da754f1417267d680"
format_stamp: "Formatted at 2026-08-11 05:15:36 on dist-test-slave-f7th"
server_key: "9f0d8e88e68c7ada1bb98b6730fcfbe1"
server_key_iv: "63545d262e9c9744b37ac76e825f2c79"
server_key_version: "encryptionkey@0"
I20260811 05:15:36.571130 21398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-2/wal/instance:
uuid: "b6e28cd99ac8431da754f1417267d680"
format_stamp: "Formatted at 2026-08-11 05:15:36 on dist-test-slave-f7th"
server_key: "9f0d8e88e68c7ada1bb98b6730fcfbe1"
server_key_iv: "63545d262e9c9744b37ac76e825f2c79"
server_key_version: "encryptionkey@0"
I20260811 05:15:36.572794 21398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260811 05:15:36.573772 21427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:36.573990 21398 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:15:36.574095 21398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-2/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-2/wal
uuid: "b6e28cd99ac8431da754f1417267d680"
format_stamp: "Formatted at 2026-08-11 05:15:36 on dist-test-slave-f7th"
server_key: "9f0d8e88e68c7ada1bb98b6730fcfbe1"
server_key_iv: "63545d262e9c9744b37ac76e825f2c79"
server_key_version: "encryptionkey@0"
I20260811 05:15:36.574175 21398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:15:36.583801 21398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:15:36.586000 21398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:15:36.586131 21398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:15:36.590315 21398 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.188:35905
I20260811 05:15:36.590373 21517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.188:35905 every 8 connection(s)
I20260811 05:15:36.590726 21398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-2/data/info.pb
I20260811 05:15:36.591681 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 21398
I20260811 05:15:36.591734 21520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:15:36.591794 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425159060450-29246-0/minicluster-data/master-2/wal/instance
I20260811 05:15:36.591964 29246 external_mini_cluster.cc:1428] Setting key b527a4a2cca650f03193a14d1ad6d1cb
I20260811 05:15:36.594190 21250 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.143.188" port: 35905 } attempt: 2
I20260811 05:15:36.594134 21520 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.143.190" port: 34367 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 05:15:36 dist-test-slave-f7th krb5kdc[21092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425336, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for kudu/127.28.143.188@KRBTEST.COM
Aug 11 05:15:36 dist-test-slave-f7th krb5kdc[21092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425336, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.188@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
Aug 11 05:15:36 dist-test-slave-f7th krb5kdc[21092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.143.188@KRBTEST.COM
I20260811 05:15:36.603113 21254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:36.594500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:35905 (local address 127.0.0.1:43524)
0811 05:15:36.594558 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:36.594569 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:36.594605 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:36.594865 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:36.594867 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:36.595090 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:15:36.595142 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:36.595144 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.596042 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.596044 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:36.596749 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:36.596751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.597291 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.597292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:36.597354 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:36.597925 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:15:36.597930 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:15:36.599449 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:15:36.601783 (+ 2334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:36.601784 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:36.601784 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:36.602053 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:36.602285 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:36.602286 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:36.602286 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:36.602345 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:36.602708 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:15:36.602709 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:15:36.602760 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 05:15:36.602895 (+ 135us) client_negotiation.cc:240] Negotiation successful
0811 05:15:36.602999 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":3}
I20260811 05:15:36.603346 21528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:36.594565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43524 (local address 127.28.143.188:35905)
0811 05:15:36.594717 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:36.594719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:36.594727 (+ 8us) server_negotiation.cc:407] Connection header received
0811 05:15:36.594746 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:36.594746 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:36.594756 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:36.595213 (+ 457us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:36.595244 (+ 31us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.595956 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.596828 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.597191 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.597546 (+ 355us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:36.599541 (+ 1995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:36.599550 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:36.599551 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:36.599558 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:36.601707 (+ 2149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:36.602125 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:36.602129 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:36.602129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:36.602226 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:36.602396 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:36.602397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:36.602397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:36.602644 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:36.602717 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:36.602970 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 05:15:36.603108 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":96,"threads_started":1}
I20260811 05:15:36.603395 21255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:36.594856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53768 (local address 127.28.143.190:34367)
0811 05:15:36.594902 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:36.594903 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:36.595111 (+ 208us) server_negotiation.cc:407] Connection header received
0811 05:15:36.595165 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:36.595166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:36.595176 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:36.595223 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:36.595684 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.596239 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.597288 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.597534 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.597574 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:36.600506 (+ 2932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:36.600508 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:36.600508 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:36.600516 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:36.602086 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:36.602575 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:36.602577 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:36.602577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:36.602637 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:36.602832 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:36.602833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:36.602834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:36.602958 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:36.603015 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:36.603249 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 05:15:36.603323 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 05:15:36.603668 21529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:36.594798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:34367 (local address 127.0.0.1:53768)
0811 05:15:36.595020 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:36.595049 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:36.595110 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:36.595252 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:36.595255 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:36.595510 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:15:36.595601 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:36.595606 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.596322 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.596323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:36.597189 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:36.597191 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.597653 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.597654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:36.597712 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:36.598113 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:15:36.598123 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:15:36.600408 (+ 2285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:15:36.602187 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:36.602195 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:36.602198 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:36.602467 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:36.602709 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:36.602710 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:36.602710 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:36.602768 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:36.603044 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:15:36.603060 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:15:36.603111 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 05:15:36.603443 (+ 332us) client_negotiation.cc:240] Negotiation successful
0811 05:15:36.603563 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":91,"threads_started":1}
I20260811 05:15:36.604257 21520 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.143.189" port: 36007 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:15:36.604534 21525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:36.592546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43518 (local address 127.28.143.188:35905)
0811 05:15:36.592723 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:36.592732 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:36.592762 (+ 30us) server_negotiation.cc:407] Connection header received
0811 05:15:36.593225 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:36.593232 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:36.593281 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:36.593403 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:36.594223 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.595612 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.596531 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.597187 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.597291 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:36.600560 (+ 3269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:36.600561 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:36.600562 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:36.600568 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:36.602995 (+ 2427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:36.603383 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:36.603384 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:36.603385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:36.603441 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:36.603635 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:36.603636 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:36.603636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:36.603754 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:36.604339 (+ 585us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:36.604377 (+ 38us) server_negotiation.cc:299] Negotiation successful
0811 05:15:36.604440 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":94,"threads_started":1}
I20260811 05:15:36.606513 21250 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6169480bab2a4508b836503948b135db: Bootstrap starting.
I20260811 05:15:36.607355 21250 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6169480bab2a4508b836503948b135db: Neither blocks nor log segments found. Creating new log.
I20260811 05:15:36.607707 21250 log.cc:824] T 00000000000000000000000000000000 P 6169480bab2a4508b836503948b135db: Log is configured to *not* fsync() on all Append() calls
I20260811 05:15:36.608876 21250 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6169480bab2a4508b836503948b135db: No bootstrap required, opened a new log
Aug 11 05:15:36 dist-test-slave-f7th krb5kdc[21092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425336, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.188@KRBTEST.COM for kudu/127.28.143.189@KRBTEST.COM
Aug 11 05:15:36 dist-test-slave-f7th krb5kdc[21092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:15:36.610754 21250 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6169480bab2a4508b836503948b135db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6169480bab2a4508b836503948b135db" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 34367 } } peers { permanent_uuid: "3845ec660a6f4c6d8185b2cabb589a37" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 36007 } } peers { permanent_uuid: "b6e28cd99ac8431da754f1417267d680" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 35905 } }
I20260811 05:15:36.610936 21250 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6169480bab2a4508b836503948b135db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:15:36.611020 21250 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6169480bab2a4508b836503948b135db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6169480bab2a4508b836503948b135db, State: Initialized, Role: FOLLOWER
I20260811 05:15:36.611172 21250 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6169480bab2a4508b836503948b135db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6169480bab2a4508b836503948b135db" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 34367 } } peers { permanent_uuid: "3845ec660a6f4c6d8185b2cabb589a37" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 36007 } } peers { permanent_uuid: "b6e28cd99ac8431da754f1417267d680" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 35905 } }
I20260811 05:15:36.611547 21250 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6169480bab2a4508b836503948b135db [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:15:36.611668 21531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6169480bab2a4508b836503948b135db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6169480bab2a4508b836503948b135db" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 34367 } } peers { permanent_uuid: "3845ec660a6f4c6d8185b2cabb589a37" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 36007 } } peers { permanent_uuid: "b6e28cd99ac8431da754f1417267d680" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 35905 } } }
I20260811 05:15:36.611819 21531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6169480bab2a4508b836503948b135db [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:15:36.612550 21392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:36.604607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51772 (local address 127.28.143.189:36007)
0811 05:15:36.604689 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:36.604691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:36.604699 (+ 8us) server_negotiation.cc:407] Connection header received
0811 05:15:36.604724 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:36.604725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:36.604735 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:36.604787 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:36.605104 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.606089 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.606954 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.607264 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.607343 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:36.609700 (+ 2357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:36.609702 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:36.609702 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:36.609712 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:36.611326 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:36.611742 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:36.611744 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:36.611744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:36.611804 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:36.612008 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:36.612009 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:36.612009 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:36.612147 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:36.612189 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:36.612387 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 05:15:36.612455 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 05:15:36.612560 21529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:36.604511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.189:36007 (local address 127.0.0.1:51772)
0811 05:15:36.604559 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:36.604568 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:36.604598 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:36.604838 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:36.604839 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:36.604963 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:15:36.605013 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:36.605015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.606178 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.606179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:36.606825 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:36.606829 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.607359 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.607360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:36.607439 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:36.608061 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:15:36.608067 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:15:36.609592 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:15:36.611418 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:36.611419 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:36.611419 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:36.611644 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:36.611878 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:36.611879 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:36.611879 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:36.611938 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:36.612236 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:15:36.612238 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:15:36.612284 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 05:15:36.612345 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 05:15:36.612453 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260811 05:15:36.613148 21520 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.143.188" port: 35905 } has no permanent_uuid. Determining permanent_uuid...
W20260811 05:15:36.613729 21547 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6169480bab2a4508b836503948b135db: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:15:36.613821 21547 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:15:36.613504 21255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:36.605116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53778 (local address 127.28.143.190:34367)
0811 05:15:36.605163 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:36.605165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:36.605172 (+ 7us) server_negotiation.cc:407] Connection header received
0811 05:15:36.605191 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:36.605192 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:36.605201 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:36.605258 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:36.606149 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.606939 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.607744 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.608008 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.608079 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:36.610583 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:36.610585 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:36.610585 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:36.610593 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:36.612106 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:36.612570 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:36.612571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:36.612572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:36.612659 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:36.612874 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:36.612875 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:36.612875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:36.612998 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:36.613113 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:36.613339 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 05:15:36.613422 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260811 05:15:36.615310 21110 master_runner.cc:186] Error getting master registration for 127.28.143.189:36007: OK, instance_id { permanent_uuid: "3845ec660a6f4c6d8185b2cabb589a37" instance_seqno: 1786425336420047 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
Aug 11 05:15:36 dist-test-slave-f7th krb5kdc[21092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425336, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.188@KRBTEST.COM for kudu/127.28.143.188@KRBTEST.COM
I20260811 05:15:36.618638 21110 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 11 05:15:36 dist-test-slave-f7th krb5kdc[21092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.143.189@KRBTEST.COM
I20260811 05:15:36.621803 21529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:36.613558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:35905 (local address 127.0.0.1:43532)
0811 05:15:36.613602 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:36.613614 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:36.613659 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:36.613788 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:36.613789 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:36.613934 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:15:36.613976 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:36.613979 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.615192 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.615193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:36.616143 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:36.616147 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.616867 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.616868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:36.616919 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:36.617372 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:15:36.617377 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:15:36.618980 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:15:36.620499 (+ 1519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:36.620500 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:36.620501 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:36.620785 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:36.621033 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:36.621034 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:36.621034 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:36.621102 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:36.621482 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:15:36.621483 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:15:36.621522 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 05:15:36.621591 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 05:15:36.621696 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260811 05:15:36.622217 21525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:36.613646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43532 (local address 127.28.143.188:35905)
0811 05:15:36.613684 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:36.613686 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:36.613694 (+ 8us) server_negotiation.cc:407] Connection header received
0811 05:15:36.613714 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:36.613715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:36.613724 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:36.613772 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:36.614060 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.614765 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.616239 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.616720 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.616783 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:36.619086 (+ 2303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:36.619088 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:36.619088 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:36.619096 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:36.620413 (+ 1317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:36.620889 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:36.620890 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:36.620891 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:36.620962 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:36.621176 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:36.621177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:36.621178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:36.621388 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:36.621439 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:36.621661 (+ 222us) server_negotiation.cc:299] Negotiation successful
0811 05:15:36.621812 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":20}
I20260811 05:15:36.623728 21392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:36.614675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51784 (local address 127.28.143.189:36007)
0811 05:15:36.614715 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:36.614717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:36.614754 (+ 37us) server_negotiation.cc:407] Connection header received
0811 05:15:36.614809 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:36.614810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:36.614818 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:36.614863 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:36.615345 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.616136 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.617467 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.617751 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.617812 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:36.620391 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:36.620393 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:36.620394 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:36.620402 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:36.622245 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:36.622837 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:36.622839 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:36.622840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:36.622913 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:36.623140 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:36.623140 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:36.623141 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:36.623290 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:36.623506 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:36.623545 (+ 39us) server_negotiation.cc:299] Negotiation successful
0811 05:15:36.623632 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 05:15:36.623881 21520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6e28cd99ac8431da754f1417267d680: Bootstrap starting.
I20260811 05:15:36.624775 21520 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6e28cd99ac8431da754f1417267d680: Neither blocks nor log segments found. Creating new log.
I20260811 05:15:36.625123 21520 log.cc:824] T 00000000000000000000000000000000 P b6e28cd99ac8431da754f1417267d680: Log is configured to *not* fsync() on all Append() calls
I20260811 05:15:36.625963 29246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:15:36.626173 21520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6e28cd99ac8431da754f1417267d680: No bootstrap required, opened a new log
I20260811 05:15:36.628423 21520 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6e28cd99ac8431da754f1417267d680 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6169480bab2a4508b836503948b135db" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 34367 } } peers { permanent_uuid: "3845ec660a6f4c6d8185b2cabb589a37" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 36007 } } peers { permanent_uuid: "b6e28cd99ac8431da754f1417267d680" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 35905 } }
I20260811 05:15:36.628788 21520 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6e28cd99ac8431da754f1417267d680 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:15:36.628880 21520 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6e28cd99ac8431da754f1417267d680 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6e28cd99ac8431da754f1417267d680, State: Initialized, Role: FOLLOWER
I20260811 05:15:36.629091 21520 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6e28cd99ac8431da754f1417267d680 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6169480bab2a4508b836503948b135db" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 34367 } } peers { permanent_uuid: "3845ec660a6f4c6d8185b2cabb589a37" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 36007 } } peers { permanent_uuid: "b6e28cd99ac8431da754f1417267d680" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 35905 } }
I20260811 05:15:36.629500 21520 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6e28cd99ac8431da754f1417267d680 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:15:36.636477 21525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:36.627814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43538 (local address 127.28.143.188:35905)
0811 05:15:36.627851 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:36.627853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:36.628177 (+ 324us) server_negotiation.cc:407] Connection header received
0811 05:15:36.628421 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:36.628422 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:36.628432 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:36.628479 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:36.629676 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.630391 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.631337 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.631682 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.631751 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:36.633375 (+ 1624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:36.633377 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:36.633377 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:36.633385 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:36.635080 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:36.635475 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:36.635476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:36.635477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:36.635532 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:36.635785 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:36.635786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:36.635786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:36.635921 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:36.636008 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:36.636273 (+ 265us) server_negotiation.cc:299] Negotiation successful
0811 05:15:36.636379 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 05:15:36.633327 21398 master_runner.cc:186] Error getting master registration for 127.28.143.189:36007: OK, instance_id { permanent_uuid: "3845ec660a6f4c6d8185b2cabb589a37" instance_seqno: 1786425336420047 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 05:15:36.637631 21398 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 05:15:36.637760 21392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:36.627650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51800 (local address 127.28.143.189:36007)
0811 05:15:36.627692 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:36.627694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:36.628079 (+ 385us) server_negotiation.cc:407] Connection header received
0811 05:15:36.628315 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:36.628316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:36.628327 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:36.628356 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:36.631346 (+ 2990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.631962 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.632940 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.633204 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.633244 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:36.634665 (+ 1421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:36.634667 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:36.634667 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:36.634676 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:36.636237 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:36.636705 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:36.636707 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:36.636707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:36.636773 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:36.636989 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:36.636990 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:36.636991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:36.637244 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:36.637321 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:36.637508 (+ 187us) server_negotiation.cc:299] Negotiation successful
0811 05:15:36.637600 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 05:15:36.640743 21581 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6e28cd99ac8431da754f1417267d680 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6169480bab2a4508b836503948b135db" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 34367 } } peers { permanent_uuid: "3845ec660a6f4c6d8185b2cabb589a37" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 36007 } } peers { permanent_uuid: "b6e28cd99ac8431da754f1417267d680" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 35905 } } }
I20260811 05:15:36.640897 21581 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6e28cd99ac8431da754f1417267d680 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:15:36.641820 21255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:36.627451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53786 (local address 127.28.143.190:34367)
0811 05:15:36.627494 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:36.627496 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:36.627955 (+ 459us) server_negotiation.cc:407] Connection header received
0811 05:15:36.627992 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:36.627993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:36.628007 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:36.628060 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:36.628863 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.629612 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.630609 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.630904 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.630945 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:36.639240 (+ 8295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:36.639241 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:36.639242 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:36.639250 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:36.640604 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:36.641068 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:36.641070 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:36.641070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:36.641129 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:36.641318 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:36.641319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:36.641319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:36.641440 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:36.641564 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:36.641649 (+ 85us) server_negotiation.cc:299] Negotiation successful
0811 05:15:36.641728 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260811 05:15:36.641829 21599 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b6e28cd99ac8431da754f1417267d680: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:15:36.641875 21599 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:15:36.660136 21384 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.143.188" port: 35905 } attempt: 3
Aug 11 05:15:36 dist-test-slave-f7th krb5kdc[21092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425336, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.189@KRBTEST.COM for kudu/127.28.143.188@KRBTEST.COM
I20260811 05:15:36.666925 21390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:36.660507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:35905 (local address 127.0.0.1:43542)
0811 05:15:36.660573 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:36.660585 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:36.660661 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:36.660795 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:36.660796 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:36.660999 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:15:36.661045 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:36.661047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.661790 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.661792 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:36.662479 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:36.662482 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.662791 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.662792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:36.662841 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:36.663261 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:15:36.663266 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:15:36.664470 (+ 1204us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:15:36.665958 (+ 1488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:36.665959 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:36.665959 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:36.666131 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:36.666304 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:36.666305 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:36.666305 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:36.666366 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:36.666590 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:15:36.666591 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:15:36.666639 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 05:15:36.666698 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 05:15:36.666804 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 05:15:36.666918 21525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:36.660555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43542 (local address 127.28.143.188:35905)
0811 05:15:36.660598 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:36.660599 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:36.660654 (+ 55us) server_negotiation.cc:407] Connection header received
0811 05:15:36.660725 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:36.660726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:36.660737 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:36.660780 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:36.661120 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.661696 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.662553 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:36.662731 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:36.662786 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:36.664555 (+ 1769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:36.664556 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:36.664556 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:36.664565 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:36.665871 (+ 1306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:36.666194 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:36.666195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:36.666195 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:36.666249 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:36.666418 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:36.666419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:36.666420 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:36.666528 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:36.666584 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:36.666727 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 05:15:36.666804 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 05:15:36.668428 21384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3845ec660a6f4c6d8185b2cabb589a37: Bootstrap starting.
I20260811 05:15:36.669276 21384 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3845ec660a6f4c6d8185b2cabb589a37: Neither blocks nor log segments found. Creating new log.
I20260811 05:15:36.669631 21384 log.cc:824] T 00000000000000000000000000000000 P 3845ec660a6f4c6d8185b2cabb589a37: Log is configured to *not* fsync() on all Append() calls
I20260811 05:15:36.670444 21384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3845ec660a6f4c6d8185b2cabb589a37: No bootstrap required, opened a new log
I20260811 05:15:36.671870 21384 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3845ec660a6f4c6d8185b2cabb589a37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6169480bab2a4508b836503948b135db" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 34367 } } peers { permanent_uuid: "3845ec660a6f4c6d8185b2cabb589a37" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 36007 } } peers { permanent_uuid: "b6e28cd99ac8431da754f1417267d680" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 35905 } }
I20260811 05:15:36.671998 21384 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3845ec660a6f4c6d8185b2cabb589a37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:15:36.672034 21384 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3845ec660a6f4c6d8185b2cabb589a37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3845ec660a6f4c6d8185b2cabb589a37, State: Initialized, Role: FOLLOWER
I20260811 05:15:36.672163 21384 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3845ec660a6f4c6d8185b2cabb589a37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6169480bab2a4508b836503948b135db" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 34367 } } peers { permanent_uuid: "3845ec660a6f4c6d8185b2cabb589a37" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 36007 } } peers { permanent_uuid: "b6e28cd99ac8431da754f1417267d680" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 35905 } }
I20260811 05:15:36.672447 21605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3845ec660a6f4c6d8185b2cabb589a37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6169480bab2a4508b836503948b135db" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 34367 } } peers { permanent_uuid: "3845ec660a6f4c6d8185b2cabb589a37" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 36007 } } peers { permanent_uuid: "b6e28cd99ac8431da754f1417267d680" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 35905 } } }
I20260811 05:15:36.672511 21384 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3845ec660a6f4c6d8185b2cabb589a37 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:15:36.672590 21605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3845ec660a6f4c6d8185b2cabb589a37 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 05:15:36.674327 21624 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3845ec660a6f4c6d8185b2cabb589a37: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:15:36.674451 21624 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:15:36.676823 21259 master_runner.cc:263] No leader master found from master 3845ec660a6f4c6d8185b2cabb589a37
I20260811 05:15:36.676884 21259 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 05:15:36.745213 21148 tablet.cc:2369] T 00000000000000000000000000000000 P 6169480bab2a4508b836503948b135db: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:15:36.766273 21531 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6169480bab2a4508b836503948b135db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:15:36.766552 21531 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6169480bab2a4508b836503948b135db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6169480bab2a4508b836503948b135db" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 34367 } } peers { permanent_uuid: "3845ec660a6f4c6d8185b2cabb589a37" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 36007 } } peers { permanent_uuid: "b6e28cd99ac8431da754f1417267d680" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 35905 } }
I20260811 05:15:36.766875 21531 leader_election.cc:288] T 00000000000000000000000000000000 P 6169480bab2a4508b836503948b135db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3845ec660a6f4c6d8185b2cabb589a37 (127.28.143.189:36007), b6e28cd99ac8431da754f1417267d680 (127.28.143.188:35905)
I20260811 05:15:36.767056 21581 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6e28cd99ac8431da754f1417267d680 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:15:36.767168 21581 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6e28cd99ac8431da754f1417267d680 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6169480bab2a4508b836503948b135db" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 34367 } } peers { permanent_uuid: "3845ec660a6f4c6d8185b2cabb589a37" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 36007 } } peers { permanent_uuid: "b6e28cd99ac8431da754f1417267d680" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 35905 } }
I20260811 05:15:36.767401 21581 leader_election.cc:288] T 00000000000000000000000000000000 P b6e28cd99ac8431da754f1417267d680 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6169480bab2a4508b836503948b135db (127.28.143.190:34367), 3845ec660a6f4c6d8185b2cabb589a37 (127.28.143.189:36007)
I20260811 05:15:36.767815 21334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b6e28cd99ac8431da754f1417267d680" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3845ec660a6f4c6d8185b2cabb589a37" is_pre_election: true
I20260811 05:15:36.767980 21334 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3845ec660a6f4c6d8185b2cabb589a37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6e28cd99ac8431da754f1417267d680 in term 0.
I20260811 05:15:36.768261 21433 leader_election.cc:302] T 00000000000000000000000000000000 P b6e28cd99ac8431da754f1417267d680 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3845ec660a6f4c6d8185b2cabb589a37, b6e28cd99ac8431da754f1417267d680; no voters:
I20260811 05:15:36.768417 21581 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6e28cd99ac8431da754f1417267d680 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:15:36.768473 21581 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6e28cd99ac8431da754f1417267d680 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:15:36.768510 21581 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6e28cd99ac8431da754f1417267d680 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:15:36.769519 21581 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6e28cd99ac8431da754f1417267d680 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6169480bab2a4508b836503948b135db" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 34367 } } peers { permanent_uuid: "3845ec660a6f4c6d8185b2cabb589a37" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 36007 } } peers { permanent_uuid: "b6e28cd99ac8431da754f1417267d680" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 35905 } }
I20260811 05:15:36.769662 21581 leader_election.cc:288] T 00000000000000000000000000000000 P b6e28cd99ac8431da754f1417267d680 [CANDIDATE]: Term 1 election: Requested vote from peers 6169480bab2a4508b836503948b135db (127.28.143.190:34367), 3845ec660a6f4c6d8185b2cabb589a37 (127.28.143.189:36007)
I20260811 05:15:36.769927 21334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b6e28cd99ac8431da754f1417267d680" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3845ec660a6f4c6d8185b2cabb589a37"
I20260811 05:15:36.770016 21334 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3845ec660a6f4c6d8185b2cabb589a37 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:15:36.770951 21334 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3845ec660a6f4c6d8185b2cabb589a37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6e28cd99ac8431da754f1417267d680 in term 1.
I20260811 05:15:36.771147 21433 leader_election.cc:302] T 00000000000000000000000000000000 P b6e28cd99ac8431da754f1417267d680 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3845ec660a6f4c6d8185b2cabb589a37, b6e28cd99ac8431da754f1417267d680; no voters:
I20260811 05:15:36.771534 21191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b6e28cd99ac8431da754f1417267d680" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6169480bab2a4508b836503948b135db" is_pre_election: true
I20260811 05:15:36.771701 21191 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6169480bab2a4508b836503948b135db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6e28cd99ac8431da754f1417267d680 in term 0.
I20260811 05:15:36.771843 21190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b6e28cd99ac8431da754f1417267d680" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6169480bab2a4508b836503948b135db"
I20260811 05:15:36.771908 21190 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6169480bab2a4508b836503948b135db [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:15:36.772912 21484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6169480bab2a4508b836503948b135db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6e28cd99ac8431da754f1417267d680" is_pre_election: true
I20260811 05:15:36.773083 21484 raft_consensus.cc:2379] T 00000000000000000000000000000000 P b6e28cd99ac8431da754f1417267d680 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6169480bab2a4508b836503948b135db in current term 1: Already voted for candidate b6e28cd99ac8431da754f1417267d680 in this term.
I20260811 05:15:36.773669 21190 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6169480bab2a4508b836503948b135db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6e28cd99ac8431da754f1417267d680 in term 1.
I20260811 05:15:36.774055 21334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6169480bab2a4508b836503948b135db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3845ec660a6f4c6d8185b2cabb589a37" is_pre_election: true
I20260811 05:15:36.774168 21334 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 3845ec660a6f4c6d8185b2cabb589a37 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6169480bab2a4508b836503948b135db in current term 1: Already voted for candidate b6e28cd99ac8431da754f1417267d680 in this term.
I20260811 05:15:36.774401 21144 leader_election.cc:302] T 00000000000000000000000000000000 P 6169480bab2a4508b836503948b135db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6169480bab2a4508b836503948b135db; no voters: 3845ec660a6f4c6d8185b2cabb589a37, b6e28cd99ac8431da754f1417267d680
I20260811 05:15:36.774602 21531 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6169480bab2a4508b836503948b135db [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 05:15:36.774703 21581 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6e28cd99ac8431da754f1417267d680 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:15:36.774878 21581 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6e28cd99ac8431da754f1417267d680 [term 1 LEADER]: Becoming Leader. State: Replica: b6e28cd99ac8431da754f1417267d680, State: Running, Role: LEADER
I20260811 05:15:36.774988 21581 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6e28cd99ac8431da754f1417267d680 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6169480bab2a4508b836503948b135db" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 34367 } } peers { permanent_uuid: "3845ec660a6f4c6d8185b2cabb589a37" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 36007 } } peers { permanent_uuid: "b6e28cd99ac8431da754f1417267d680" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 35905 } }
I20260811 05:15:36.775422 21581 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6e28cd99ac8431da754f1417267d680 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6e28cd99ac8431da754f1417267d680. Latest consensus state: current_term: 1 leader_uuid: "b6e28cd99ac8431da754f1417267d680" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6169480bab2a4508b836503948b135db" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 34367 } } peers { permanent_uuid: "3845ec660a6f4c6d8185b2cabb589a37" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 36007 } } peers { permanent_uuid: "b6e28cd99ac8431da754f1417267d680" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 35905 } } }
I20260811 05:15:36.775648 21581 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6e28cd99ac8431da754f1417267d680 [sys.catalog]: This master's current role is: LEADER
I20260811 05:15:36.775981 21630 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:15:36.776193 21630 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:15:36.779722 21334 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3845ec660a6f4c6d8185b2cabb589a37 [term 1 FOLLOWER]: Refusing update from remote peer b6e28cd99ac8431da754f1417267d680: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:15:36.780058 21581 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b6e28cd99ac8431da754f1417267d680 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3845ec660a6f4c6d8185b2cabb589a37" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 36007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:15:36.781788 21605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3845ec660a6f4c6d8185b2cabb589a37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6e28cd99ac8431da754f1417267d680. Latest consensus state: current_term: 1 leader_uuid: "b6e28cd99ac8431da754f1417267d680" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6169480bab2a4508b836503948b135db" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 34367 } } peers { permanent_uuid: "3845ec660a6f4c6d8185b2cabb589a37" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 36007 } } peers { permanent_uuid: "b6e28cd99ac8431da754f1417267d680" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 35905 } } }
I20260811 05:15:36.781904 21605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3845ec660a6f4c6d8185b2cabb589a37 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:15:36.782361 21190 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6169480bab2a4508b836503948b135db [term 1 FOLLOWER]: Refusing update from remote peer b6e28cd99ac8431da754f1417267d680: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:15:36.782687 21581 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b6e28cd99ac8431da754f1417267d680 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6169480bab2a4508b836503948b135db" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 34367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:15:36.784000 21531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6169480bab2a4508b836503948b135db [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6e28cd99ac8431da754f1417267d680. Latest consensus state: current_term: 1 leader_uuid: "b6e28cd99ac8431da754f1417267d680" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6169480bab2a4508b836503948b135db" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 34367 } } peers { permanent_uuid: "3845ec660a6f4c6d8185b2cabb589a37" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 36007 } } peers { permanent_uuid: "b6e28cd99ac8431da754f1417267d680" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 35905 } } }
I20260811 05:15:36.784107 21531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6169480bab2a4508b836503948b135db [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:15:36.785570 21627 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6e28cd99ac8431da754f1417267d680 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b6e28cd99ac8431da754f1417267d680" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6169480bab2a4508b836503948b135db" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 34367 } } peers { permanent_uuid: "3845ec660a6f4c6d8185b2cabb589a37" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 36007 } } peers { permanent_uuid: "b6e28cd99ac8431da754f1417267d680" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 35905 } } }
I20260811 05:15:36.785674 21627 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6e28cd99ac8431da754f1417267d680 [sys.catalog]: This master's current role is: LEADER
I20260811 05:15:36.786190 21531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6169480bab2a4508b836503948b135db [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b6e28cd99ac8431da754f1417267d680" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6169480bab2a4508b836503948b135db" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 34367 } } peers { permanent_uuid: "3845ec660a6f4c6d8185b2cabb589a37" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 36007 } } peers { permanent_uuid: "b6e28cd99ac8431da754f1417267d680" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 35905 } } }
I20260811 05:15:36.786293 21531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6169480bab2a4508b836503948b135db [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:15:36.786521 21633 mvcc.cc:205] Tried to move back new op lower bound from 7317198179446730752 to 7317198179431043072. Current Snapshot: MvccSnapshot[applied={T|T < 7317198179446730752}]
I20260811 05:15:36.786710 21639 mvcc.cc:205] Tried to move back new op lower bound from 7317198179446730752 to 7317198179431043072. Current Snapshot: MvccSnapshot[applied={T|T < 7317198179446730752}]
I20260811 05:15:36.787595 21581 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6e28cd99ac8431da754f1417267d680 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b6e28cd99ac8431da754f1417267d680" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6169480bab2a4508b836503948b135db" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 34367 } } peers { permanent_uuid: "3845ec660a6f4c6d8185b2cabb589a37" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 36007 } } peers { permanent_uuid: "b6e28cd99ac8431da754f1417267d680" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 35905 } } }
I20260811 05:15:36.787691 21581 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6e28cd99ac8431da754f1417267d680 [sys.catalog]: This master's current role is: LEADER
I20260811 05:15:36.788074 21605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3845ec660a6f4c6d8185b2cabb589a37 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b6e28cd99ac8431da754f1417267d680" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6169480bab2a4508b836503948b135db" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 34367 } } peers { permanent_uuid: "3845ec660a6f4c6d8185b2cabb589a37" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 36007 } } peers { permanent_uuid: "b6e28cd99ac8431da754f1417267d680" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 35905 } } }
I20260811 05:15:36.788177 21605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3845ec660a6f4c6d8185b2cabb589a37 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:15:36.788939 21636 mvcc.cc:205] Tried to move back new op lower bound from 7317198179446730752 to 7317198179431043072. Current Snapshot: MvccSnapshot[applied={T|T < 7317198179446730752}]
I20260811 05:15:36.789995 21630 catalog_manager.cc:1347] Generated new cluster ID: 0fb553fe20f64161ac9894cc72ebe052
I20260811 05:15:36.790059 21630 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:15:36.819561 21630 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:15:36.820333 21630 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:15:36.831489 21630 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6e28cd99ac8431da754f1417267d680: Generated new TSK 0
I20260811 05:15:36.831732 21630 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:15:37.614359 21547 catalog_manager.cc:1259] Loaded cluster ID: 0fb553fe20f64161ac9894cc72ebe052
I20260811 05:15:37.614445 21547 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6169480bab2a4508b836503948b135db: loading cluster ID for follower catalog manager: success
I20260811 05:15:37.615237 21547 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6169480bab2a4508b836503948b135db: acquiring CA information for follower catalog manager: success
I20260811 05:15:37.615504 21547 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6169480bab2a4508b836503948b135db: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 05:15:37.674978 21624 catalog_manager.cc:1259] Loaded cluster ID: 0fb553fe20f64161ac9894cc72ebe052
I20260811 05:15:37.675064 21624 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3845ec660a6f4c6d8185b2cabb589a37: loading cluster ID for follower catalog manager: success
I20260811 05:15:37.675943 21624 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3845ec660a6f4c6d8185b2cabb589a37: acquiring CA information for follower catalog manager: success
I20260811 05:15:37.676280 21624 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3845ec660a6f4c6d8185b2cabb589a37: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 05:15:37.884579 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 21110
I20260811 05:15:37.896482 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 21259
I20260811 05:15:37.909307 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 21398
2026-08-11T05:15:37Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3135 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425159060450-29246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:15:37 dist-test-slave-f7th krb5kdc[21654](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:15:37 dist-test-slave-f7th krb5kdc[21654](info): set up 2 sockets
Aug 11 05:15:37 dist-test-slave-f7th krb5kdc[21654](info): commencing operation
krb5kdc: starting...
W20260811 05:15:39.114048 29246 mini_kdc.cc:121] Time spent starting KDC: real 1.179s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Aug 11 05:15:39 dist-test-slave-f7th krb5kdc[21654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:15:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:15:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
I20260811 05:15:39.256031 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:43279
--webserver_interface=127.28.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:39523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:43279
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:15:39.370988 21678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:15:39.371232 21678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:15:39.371256 21678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:15:39.372562 21678 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:15:39.372653 21678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:15:39.372669 21678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:15:39.372694 21678 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:15:39.372711 21678 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:15:39.374517 21678 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:39523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:43279
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:43279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21678
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:15:39.374815 21678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:15:39.375145 21678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:15:39.380517 21688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:39.380507 21683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:39.380607 21678 server_base.cc:1034] running on GCE node
W20260811 05:15:39.380549 21686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:39.381278 21678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:15:39.381598 21678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:15:39.382782 21678 hybrid_clock.cc:648] HybridClock initialized: now 1786425339382765 us; error 32 us; skew 500 ppm
Aug 11 05:15:39 dist-test-slave-f7th krb5kdc[21654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425339, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:15:39.385604 21678 init.cc:377] Logged in from keytab as kudu/127.28.143.190@KRBTEST.COM (short username kudu)
I20260811 05:15:39.386078 21678 webserver.cc:460] Webserver started at http://127.28.143.190:41577/ using document root <none> and password file <none>
I20260811 05:15:39.386319 21678 fs_manager.cc:359] Metadata directory not provided
I20260811 05:15:39.386365 21678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:15:39.386482 21678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:15:39.387606 21678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "ca003442ba954befb980c50dd18de73f"
format_stamp: "Formatted at 2026-08-11 05:15:39 on dist-test-slave-f7th"
server_key: "96de666f9288844d48de6023e4d2e538"
server_key_iv: "5895fb40ec1c747e8eea79e8bdb9824a"
server_key_version: "encryptionkey@0"
I20260811 05:15:39.387961 21678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "ca003442ba954befb980c50dd18de73f"
format_stamp: "Formatted at 2026-08-11 05:15:39 on dist-test-slave-f7th"
server_key: "96de666f9288844d48de6023e4d2e538"
server_key_iv: "5895fb40ec1c747e8eea79e8bdb9824a"
server_key_version: "encryptionkey@0"
I20260811 05:15:39.389494 21678 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 05:15:39.390395 21699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:39.390609 21678 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:15:39.390715 21678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "ca003442ba954befb980c50dd18de73f"
format_stamp: "Formatted at 2026-08-11 05:15:39 on dist-test-slave-f7th"
server_key: "96de666f9288844d48de6023e4d2e538"
server_key_iv: "5895fb40ec1c747e8eea79e8bdb9824a"
server_key_version: "encryptionkey@0"
I20260811 05:15:39.390807 21678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:15:39.414746 21678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:15:39.417294 21678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:15:39.417435 21678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:15:39.421680 21678 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:43279
I20260811 05:15:39.421725 21780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:43279 every 8 connection(s)
I20260811 05:15:39.422086 21678 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:15:39.423095 21783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:15:39.425618 21783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca003442ba954befb980c50dd18de73f: Bootstrap starting.
I20260811 05:15:39.426266 21783 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca003442ba954befb980c50dd18de73f: Neither blocks nor log segments found. Creating new log.
I20260811 05:15:39.426550 21783 log.cc:824] T 00000000000000000000000000000000 P ca003442ba954befb980c50dd18de73f: Log is configured to *not* fsync() on all Append() calls
I20260811 05:15:39.427345 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 21678
I20260811 05:15:39.427448 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:15:39.427644 29246 external_mini_cluster.cc:1428] Setting key bcf44c45b8a2ae6762f44a09cef8cf12
I20260811 05:15:39.427996 21783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca003442ba954befb980c50dd18de73f: No bootstrap required, opened a new log
I20260811 05:15:39.431190 21783 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca003442ba954befb980c50dd18de73f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca003442ba954befb980c50dd18de73f" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 43279 } }
I20260811 05:15:39.431387 21783 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca003442ba954befb980c50dd18de73f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:15:39.431417 21783 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca003442ba954befb980c50dd18de73f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca003442ba954befb980c50dd18de73f, State: Initialized, Role: FOLLOWER
I20260811 05:15:39.431545 21783 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca003442ba954befb980c50dd18de73f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca003442ba954befb980c50dd18de73f" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 43279 } }
I20260811 05:15:39.431624 21783 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca003442ba954befb980c50dd18de73f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:15:39.431651 21783 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca003442ba954befb980c50dd18de73f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:15:39.431697 21783 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca003442ba954befb980c50dd18de73f [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:15:39.432401 21783 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca003442ba954befb980c50dd18de73f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca003442ba954befb980c50dd18de73f" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 43279 } }
I20260811 05:15:39.432520 21783 leader_election.cc:302] T 00000000000000000000000000000000 P ca003442ba954befb980c50dd18de73f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca003442ba954befb980c50dd18de73f; no voters:
I20260811 05:15:39.432736 21783 leader_election.cc:288] T 00000000000000000000000000000000 P ca003442ba954befb980c50dd18de73f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:15:39.432773 21791 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca003442ba954befb980c50dd18de73f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:15:39.432948 21791 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca003442ba954befb980c50dd18de73f [term 1 LEADER]: Becoming Leader. State: Replica: ca003442ba954befb980c50dd18de73f, State: Running, Role: LEADER
I20260811 05:15:39.433035 21783 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca003442ba954befb980c50dd18de73f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:15:39.433099 21791 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca003442ba954befb980c50dd18de73f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca003442ba954befb980c50dd18de73f" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 43279 } }
I20260811 05:15:39.433465 21791 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca003442ba954befb980c50dd18de73f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca003442ba954befb980c50dd18de73f. Latest consensus state: current_term: 1 leader_uuid: "ca003442ba954befb980c50dd18de73f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca003442ba954befb980c50dd18de73f" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 43279 } } }
I20260811 05:15:39.433540 21791 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca003442ba954befb980c50dd18de73f [sys.catalog]: This master's current role is: LEADER
I20260811 05:15:39.433491 21792 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca003442ba954befb980c50dd18de73f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca003442ba954befb980c50dd18de73f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca003442ba954befb980c50dd18de73f" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 43279 } } }
I20260811 05:15:39.433604 21792 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca003442ba954befb980c50dd18de73f [sys.catalog]: This master's current role is: LEADER
W20260811 05:15:39.434973 21821 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ca003442ba954befb980c50dd18de73f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:15:39.435075 21821 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:15:39.435176 21803 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:15:39.435413 21803 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 11 05:15:39 dist-test-slave-f7th krb5kdc[21654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:15:39.437150 21803 catalog_manager.cc:1347] Generated new cluster ID: 03ea3e8eb54543d482f5978a46e24ca5
I20260811 05:15:39.437243 21803 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:15:39.439464 21790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:39.428132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49474 (local address 127.28.143.190:43279)
0811 05:15:39.428307 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:39.428318 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:39.428356 (+ 38us) server_negotiation.cc:407] Connection header received
0811 05:15:39.428958 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:39.428969 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:39.429040 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:39.429197 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:39.429813 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:39.431227 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:39.432130 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:39.432784 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:39.432907 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:39.436100 (+ 3193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:39.436105 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:39.436106 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:39.436119 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:39.437959 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:39.438314 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:39.438317 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:39.438317 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:39.438405 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:39.438592 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:39.438593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:39.438593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:39.438843 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:39.438997 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:39.439114 (+ 117us) server_negotiation.cc:299] Negotiation successful
0811 05:15:39.439261 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":66,"threads_started":1}
I20260811 05:15:39.440337 29246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:15:39.452109 21803 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:15:39.452728 21803 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:15:39.459884 21803 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca003442ba954befb980c50dd18de73f: Generated new TSK 0
I20260811 05:15:39.460129 21803 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:15:39.494714 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 21678
2026-08-11T05:15:39Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1595 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:15:39 dist-test-slave-f7th krb5kdc[21829](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:15:39 dist-test-slave-f7th krb5kdc[21829](info): set up 2 sockets
Aug 11 05:15:39 dist-test-slave-f7th krb5kdc[21829](info): commencing operation
krb5kdc: starting...
W20260811 05:15:40.712658 29246 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Aug 11 05:15:40 dist-test-slave-f7th krb5kdc[21829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:15:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:15:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
I20260811 05:15:40.844913 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:43443
--webserver_interface=127.28.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:42659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:43443
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:15:40.929874 21855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:15:40.930100 21855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:15:40.930125 21855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:15:40.931332 21855 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:15:40.931382 21855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:15:40.931396 21855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:15:40.931416 21855 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:15:40.931429 21855 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:15:40.933063 21855 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:42659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:43443
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:43443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21855
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:15:40.933326 21855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:15:40.933564 21855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 05:15:40.938969 21855 server_base.cc:1034] running on GCE node
W20260811 05:15:40.938970 21862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:40.939147 21865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:40.938972 21863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:40.939498 21855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:15:40.939817 21855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:15:40.940970 21855 hybrid_clock.cc:648] HybridClock initialized: now 1786425340940951 us; error 33 us; skew 500 ppm
Aug 11 05:15:40 dist-test-slave-f7th krb5kdc[21829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425340, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:15:40.943549 21855 init.cc:377] Logged in from keytab as kudu/127.28.143.190@KRBTEST.COM (short username kudu)
I20260811 05:15:40.943988 21855 webserver.cc:460] Webserver started at http://127.28.143.190:43551/ using document root <none> and password file <none>
I20260811 05:15:40.944218 21855 fs_manager.cc:359] Metadata directory not provided
I20260811 05:15:40.944262 21855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:15:40.944360 21855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:15:40.945494 21855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "b717aaaa6c264b3f9f6dd77dd28788ff"
format_stamp: "Formatted at 2026-08-11 05:15:40 on dist-test-slave-f7th"
server_key: "d628d0324e6406adc48a51a0d75fc9d7"
server_key_iv: "d96516cb84150e99e92cb2dbe92430fb"
server_key_version: "encryptionkey@0"
I20260811 05:15:40.945806 21855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "b717aaaa6c264b3f9f6dd77dd28788ff"
format_stamp: "Formatted at 2026-08-11 05:15:40 on dist-test-slave-f7th"
server_key: "d628d0324e6406adc48a51a0d75fc9d7"
server_key_iv: "d96516cb84150e99e92cb2dbe92430fb"
server_key_version: "encryptionkey@0"
I20260811 05:15:40.947120 21855 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 05:15:40.947844 21879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:40.948040 21855 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 05:15:40.948118 21855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "b717aaaa6c264b3f9f6dd77dd28788ff"
format_stamp: "Formatted at 2026-08-11 05:15:40 on dist-test-slave-f7th"
server_key: "d628d0324e6406adc48a51a0d75fc9d7"
server_key_iv: "d96516cb84150e99e92cb2dbe92430fb"
server_key_version: "encryptionkey@0"
I20260811 05:15:40.948179 21855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:15:40.956249 21855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:15:40.958420 21855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:15:40.958547 21855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:15:40.962498 21855 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:43443
I20260811 05:15:40.962582 21957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:43443 every 8 connection(s)
I20260811 05:15:40.962881 21855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:15:40.963904 21960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:15:40.966313 21960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff: Bootstrap starting.
I20260811 05:15:40.966960 21960 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff: Neither blocks nor log segments found. Creating new log.
I20260811 05:15:40.967238 21960 log.cc:824] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff: Log is configured to *not* fsync() on all Append() calls
I20260811 05:15:40.967996 21960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff: No bootstrap required, opened a new log
I20260811 05:15:40.969720 21960 raft_consensus.cc:348] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b717aaaa6c264b3f9f6dd77dd28788ff" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 43443 } }
I20260811 05:15:40.969877 21960 raft_consensus.cc:374] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:15:40.969913 21960 raft_consensus.cc:729] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b717aaaa6c264b3f9f6dd77dd28788ff, State: Initialized, Role: FOLLOWER
I20260811 05:15:40.970068 21960 consensus_queue.cc:260] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b717aaaa6c264b3f9f6dd77dd28788ff" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 43443 } }
I20260811 05:15:40.970155 21960 raft_consensus.cc:388] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:15:40.970189 21960 raft_consensus.cc:482] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:15:40.970244 21960 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:15:40.971146 21960 raft_consensus.cc:504] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b717aaaa6c264b3f9f6dd77dd28788ff" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 43443 } }
I20260811 05:15:40.971235 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 21855
I20260811 05:15:40.971271 21960 leader_election.cc:302] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b717aaaa6c264b3f9f6dd77dd28788ff; no voters:
I20260811 05:15:40.971315 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:15:40.971454 21960 leader_election.cc:288] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:15:40.971508 29246 external_mini_cluster.cc:1428] Setting key fc02fa18644e2c87eea07b8afd75e3fd
I20260811 05:15:40.971529 21968 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:15:40.971698 21968 raft_consensus.cc:686] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [term 1 LEADER]: Becoming Leader. State: Replica: b717aaaa6c264b3f9f6dd77dd28788ff, State: Running, Role: LEADER
I20260811 05:15:40.971784 21960 sys_catalog.cc:564] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:15:40.971951 21968 consensus_queue.cc:237] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b717aaaa6c264b3f9f6dd77dd28788ff" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 43443 } }
I20260811 05:15:40.972352 21973 sys_catalog.cc:455] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader b717aaaa6c264b3f9f6dd77dd28788ff. Latest consensus state: current_term: 1 leader_uuid: "b717aaaa6c264b3f9f6dd77dd28788ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b717aaaa6c264b3f9f6dd77dd28788ff" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 43443 } } }
I20260811 05:15:40.972431 21973 sys_catalog.cc:458] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [sys.catalog]: This master's current role is: LEADER
I20260811 05:15:40.972797 21970 sys_catalog.cc:455] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b717aaaa6c264b3f9f6dd77dd28788ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b717aaaa6c264b3f9f6dd77dd28788ff" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 43443 } } }
I20260811 05:15:40.972869 21970 sys_catalog.cc:458] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [sys.catalog]: This master's current role is: LEADER
I20260811 05:15:40.973192 21980 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:15:40.973686 21980 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:15:40.975282 21980 catalog_manager.cc:1347] Generated new cluster ID: cf5ae5b924c64c05879742bfcfaf8b3a
I20260811 05:15:40.975361 21980 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 05:15:40 dist-test-slave-f7th krb5kdc[21829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:15:40.983150 21977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:40.972042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40872 (local address 127.28.143.190:43443)
0811 05:15:40.972307 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:40.972316 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:40.972350 (+ 34us) server_negotiation.cc:407] Connection header received
0811 05:15:40.972479 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:40.972486 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:40.972547 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:40.973551 (+ 1004us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:40.973670 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:40.974921 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:40.976025 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:40.976433 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:40.976522 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:40.979391 (+ 2869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:40.979396 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:40.979397 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:40.979411 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:40.981405 (+ 1994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:40.981871 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:40.981875 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:40.981875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:40.981999 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:40.982210 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:40.982210 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:40.982211 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:40.982518 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:40.982679 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:40.982784 (+ 105us) server_negotiation.cc:299] Negotiation successful
0811 05:15:40.982937 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":155,"threads_started":1}
I20260811 05:15:40.989910 21980 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:15:40.990460 21980 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:15:40.998021 21980 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff: Generated new TSK 0
I20260811 05:15:40.998270 21980 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.143.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab.
Entry for principal kudu/127.28.143.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab.
I20260811 05:15:41.032390 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.129:0
--local_ip_for_outbound_sockets=127.28.143.129
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:43443
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:42659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:15:41.121106 22004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:15:41.121351 22004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:15:41.121388 22004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:15:41.122614 22004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:15:41.122669 22004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:15:41.122707 22004 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.129
I20260811 05:15:41.124294 22004 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:42659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:43443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22004
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.129
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:15:41.124580 22004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:15:41.124837 22004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:15:41.130360 22010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:41.130390 22014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:41.130445 22012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:41.130508 22004 server_base.cc:1034] running on GCE node
I20260811 05:15:41.130808 22004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:15:41.131080 22004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:15:41.132265 22004 hybrid_clock.cc:648] HybridClock initialized: now 1786425341132247 us; error 31 us; skew 500 ppm
Aug 11 05:15:41 dist-test-slave-f7th krb5kdc[21829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425341, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:15:41.135319 22004 init.cc:377] Logged in from keytab as kudu/127.28.143.129@KRBTEST.COM (short username kudu)
I20260811 05:15:41.135911 22004 webserver.cc:460] Webserver started at http://127.28.143.129:37341/ using document root <none> and password file <none>
I20260811 05:15:41.136159 22004 fs_manager.cc:359] Metadata directory not provided
I20260811 05:15:41.136214 22004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:15:41.136348 22004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:15:41.137532 22004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/data/instance:
uuid: "1097935f8aa34158a3bda341d0425928"
format_stamp: "Formatted at 2026-08-11 05:15:41 on dist-test-slave-f7th"
server_key: "4353b587e41a7fff4f97414272035c06"
server_key_iv: "85a4f5b7dc193e53a34bd02c50395da0"
server_key_version: "encryptionkey@0"
I20260811 05:15:41.137925 22004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance:
uuid: "1097935f8aa34158a3bda341d0425928"
format_stamp: "Formatted at 2026-08-11 05:15:41 on dist-test-slave-f7th"
server_key: "4353b587e41a7fff4f97414272035c06"
server_key_iv: "85a4f5b7dc193e53a34bd02c50395da0"
server_key_version: "encryptionkey@0"
I20260811 05:15:41.139279 22004 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 05:15:41.140069 22026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:41.140272 22004 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 05:15:41.140386 22004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/wal
uuid: "1097935f8aa34158a3bda341d0425928"
format_stamp: "Formatted at 2026-08-11 05:15:41 on dist-test-slave-f7th"
server_key: "4353b587e41a7fff4f97414272035c06"
server_key_iv: "85a4f5b7dc193e53a34bd02c50395da0"
server_key_version: "encryptionkey@0"
I20260811 05:15:41.140486 22004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:15:41.157377 22004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:15:41.159498 22004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:15:41.159623 22004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:15:41.159828 22004 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:15:41.160135 22004 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:15:41.160167 22004 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:41.160189 22004 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:15:41.160207 22004 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:41.165958 22004 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.129:34427
I20260811 05:15:41.166052 22225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.129:34427 every 8 connection(s)
I20260811 05:15:41.166366 22004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
I20260811 05:15:41.169340 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 22004
I20260811 05:15:41.169462 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance
I20260811 05:15:41.169636 29246 external_mini_cluster.cc:1428] Setting key 69799fadce3055d565bd6b685829762c
Aug 11 05:15:41 dist-test-slave-f7th krb5kdc[21829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425341, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:15:41.177822 21977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:41.167226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:49225 (local address 127.28.143.190:43443)
0811 05:15:41.167322 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:41.167325 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:41.167500 (+ 175us) server_negotiation.cc:407] Connection header received
0811 05:15:41.168079 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:41.168082 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:41.168101 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:41.168164 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:41.169341 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:41.169942 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:41.171166 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:41.171498 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:41.171576 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:41.174375 (+ 2799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:41.174377 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:41.174377 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:41.174391 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:41.175898 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:41.176509 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:41.176510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:41.176511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:41.176592 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:41.177008 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:41.177009 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:41.177009 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:41.177157 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:41.177226 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:41.177642 (+ 416us) server_negotiation.cc:299] Negotiation successful
0811 05:15:41.177748 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 05:15:41.178067 22229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:41.167143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.129:49225)
0811 05:15:41.167354 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:41.167408 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:41.167961 (+ 553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:41.168243 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:41.168247 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:41.168949 (+ 702us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:15:41.169218 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:41.169225 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:41.170054 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:41.170056 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:41.171037 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:41.171041 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:41.171596 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:41.171597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:41.171733 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:41.172315 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:15:41.172325 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:15:41.174259 (+ 1934us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:15:41.176006 (+ 1747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:41.176019 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:41.176022 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:41.176400 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:41.176702 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:41.176703 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:41.176704 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:41.176902 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:41.177234 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:15:41.177245 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:15:41.177448 (+ 203us) client_negotiation.cc:769] Sending connection context
0811 05:15:41.177580 (+ 132us) client_negotiation.cc:240] Negotiation successful
0811 05:15:41.177792 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":102,"threads_started":1}
I20260811 05:15:41.178666 22226 heartbeater.cc:347] Connected to a master server at 127.28.143.190:43443
I20260811 05:15:41.178782 22226 heartbeater.cc:464] Registering TS with master...
I20260811 05:15:41.178979 22226 heartbeater.cc:511] Master 127.28.143.190:43443 requested a full tablet report, sending...
I20260811 05:15:41.179580 21903 ts_manager.cc:194] Registered new tserver with Master: 1097935f8aa34158a3bda341d0425928 (127.28.143.129:34427)
WARNING: no policy specified for kudu/127.28.143.130@KRBTEST.COM; defaulting to no policy
I20260811 05:15:41.180223 21903 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.129@KRBTEST.COM'} at 127.28.143.129:49225
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab.
Entry for principal kudu/127.28.143.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab.
I20260811 05:15:41.200172 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.130:0
--local_ip_for_outbound_sockets=127.28.143.130
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:43443
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:42659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:15:41.290736 22238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:15:41.290982 22238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:15:41.291008 22238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:15:41.292277 22238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:15:41.292333 22238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:15:41.292362 22238 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.130
I20260811 05:15:41.293972 22238 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:42659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:43443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.130
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:15:41.294231 22238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:15:41.294471 22238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:15:41.299713 22252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:41.299778 22238 server_base.cc:1034] running on GCE node
W20260811 05:15:41.299678 22246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:41.299686 22250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:41.300181 22238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:15:41.300452 22238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:15:41.301607 22238 hybrid_clock.cc:648] HybridClock initialized: now 1786425341301575 us; error 45 us; skew 500 ppm
Aug 11 05:15:41 dist-test-slave-f7th krb5kdc[21829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425341, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:15:41.308813 22238 init.cc:377] Logged in from keytab as kudu/127.28.143.130@KRBTEST.COM (short username kudu)
I20260811 05:15:41.309368 22238 webserver.cc:460] Webserver started at http://127.28.143.130:34815/ using document root <none> and password file <none>
I20260811 05:15:41.309643 22238 fs_manager.cc:359] Metadata directory not provided
I20260811 05:15:41.309697 22238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:15:41.309827 22238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:15:41.310912 22238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/data/instance:
uuid: "40fd4235fcb941ba9da073d48d2599a2"
format_stamp: "Formatted at 2026-08-11 05:15:41 on dist-test-slave-f7th"
server_key: "944e62cded5923f47f98069b91f3ced2"
server_key_iv: "6d4db8d3d27771a48e72c7fbf78d0ba4"
server_key_version: "encryptionkey@0"
I20260811 05:15:41.311242 22238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance:
uuid: "40fd4235fcb941ba9da073d48d2599a2"
format_stamp: "Formatted at 2026-08-11 05:15:41 on dist-test-slave-f7th"
server_key: "944e62cded5923f47f98069b91f3ced2"
server_key_iv: "6d4db8d3d27771a48e72c7fbf78d0ba4"
server_key_version: "encryptionkey@0"
I20260811 05:15:41.312561 22238 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260811 05:15:41.313350 22262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:41.313522 22238 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 05:15:41.313611 22238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/wal
uuid: "40fd4235fcb941ba9da073d48d2599a2"
format_stamp: "Formatted at 2026-08-11 05:15:41 on dist-test-slave-f7th"
server_key: "944e62cded5923f47f98069b91f3ced2"
server_key_iv: "6d4db8d3d27771a48e72c7fbf78d0ba4"
server_key_version: "encryptionkey@0"
I20260811 05:15:41.313673 22238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:15:41.321622 22238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:15:41.324088 22238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:15:41.324255 22238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:15:41.324515 22238 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:15:41.324949 22238 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:15:41.324991 22238 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:41.325021 22238 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:15:41.325039 22238 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:41.331161 22238 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.130:38185
I20260811 05:15:41.331211 22477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.130:38185 every 8 connection(s)
I20260811 05:15:41.331557 22238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
I20260811 05:15:41.336786 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 22238
I20260811 05:15:41.336947 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance
I20260811 05:15:41.337128 29246 external_mini_cluster.cc:1428] Setting key be6448e7c77309de55b22cb1bbd9e4f8
Aug 11 05:15:41 dist-test-slave-f7th krb5kdc[21829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425341, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:15:41.342208 21977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:41.332366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:54849 (local address 127.28.143.190:43443)
0811 05:15:41.332414 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:41.332416 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:41.332659 (+ 243us) server_negotiation.cc:407] Connection header received
0811 05:15:41.333317 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:41.333319 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:41.333339 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:41.333394 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:41.334403 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:41.334914 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:41.335913 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:41.336108 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:41.336166 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:41.338807 (+ 2641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:41.338808 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:41.338809 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:41.338818 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:41.340420 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:41.341068 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:41.341069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:41.341070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:41.341136 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:41.341493 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:41.341495 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:41.341495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:41.341631 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:41.341694 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:41.342050 (+ 356us) server_negotiation.cc:299] Negotiation successful
0811 05:15:41.342139 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 05:15:41.342414 22484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:41.332317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.130:54849)
0811 05:15:41.332516 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:41.332555 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:41.333207 (+ 652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:41.333486 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:41.333490 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:41.334089 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:15:41.334296 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:41.334300 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:41.335004 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:41.335005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:41.335817 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:41.335819 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:41.336181 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:41.336182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:41.336319 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:41.336984 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:15:41.336996 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:15:41.338702 (+ 1706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:15:41.340559 (+ 1857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:41.340571 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:41.340574 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:41.340964 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:41.341210 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:41.341211 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:41.341211 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:41.341371 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:41.341701 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:15:41.341708 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:15:41.341884 (+ 176us) client_negotiation.cc:769] Sending connection context
0811 05:15:41.341993 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 05:15:41.342169 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":101,"threads_started":1}
I20260811 05:15:41.342976 22478 heartbeater.cc:347] Connected to a master server at 127.28.143.190:43443
I20260811 05:15:41.343091 22478 heartbeater.cc:464] Registering TS with master...
I20260811 05:15:41.343324 22478 heartbeater.cc:511] Master 127.28.143.190:43443 requested a full tablet report, sending...
I20260811 05:15:41.343822 21903 ts_manager.cc:194] Registered new tserver with Master: 40fd4235fcb941ba9da073d48d2599a2 (127.28.143.130:38185)
I20260811 05:15:41.344241 21903 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.130@KRBTEST.COM'} at 127.28.143.130:54849
WARNING: no policy specified for kudu/127.28.143.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab.
Entry for principal kudu/127.28.143.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab.
I20260811 05:15:41.368412 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.131:0
--local_ip_for_outbound_sockets=127.28.143.131
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:43443
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:42659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:15:41.454816 22487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:15:41.455039 22487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:15:41.455063 22487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:15:41.456216 22487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:15:41.456259 22487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:15:41.456283 22487 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.131
I20260811 05:15:41.457834 22487 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:42659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:43443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.131
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:15:41.458070 22487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:15:41.458266 22487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:15:41.463716 22493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:41.463749 22487 server_base.cc:1034] running on GCE node
W20260811 05:15:41.463716 22497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:41.464010 22492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:41.464730 22487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:15:41.465077 22487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:15:41.466243 22487 hybrid_clock.cc:648] HybridClock initialized: now 1786425341466218 us; error 38 us; skew 500 ppm
Aug 11 05:15:41 dist-test-slave-f7th krb5kdc[21829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425341, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:15:41.469583 22487 init.cc:377] Logged in from keytab as kudu/127.28.143.131@KRBTEST.COM (short username kudu)
I20260811 05:15:41.470118 22487 webserver.cc:460] Webserver started at http://127.28.143.131:39033/ using document root <none> and password file <none>
I20260811 05:15:41.470368 22487 fs_manager.cc:359] Metadata directory not provided
I20260811 05:15:41.470419 22487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:15:41.470538 22487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:15:41.471630 22487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/data/instance:
uuid: "4add4ab031844102a37662ac5da7da58"
format_stamp: "Formatted at 2026-08-11 05:15:41 on dist-test-slave-f7th"
server_key: "0fb23a895c5291c35dcfb1f19d177927"
server_key_iv: "3ba01fce1777e16b09f90eb6886475fa"
server_key_version: "encryptionkey@0"
I20260811 05:15:41.471978 22487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance:
uuid: "4add4ab031844102a37662ac5da7da58"
format_stamp: "Formatted at 2026-08-11 05:15:41 on dist-test-slave-f7th"
server_key: "0fb23a895c5291c35dcfb1f19d177927"
server_key_iv: "3ba01fce1777e16b09f90eb6886475fa"
server_key_version: "encryptionkey@0"
I20260811 05:15:41.473564 22487 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 05:15:41.474479 22511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:41.474669 22487 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:15:41.474763 22487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/wal
uuid: "4add4ab031844102a37662ac5da7da58"
format_stamp: "Formatted at 2026-08-11 05:15:41 on dist-test-slave-f7th"
server_key: "0fb23a895c5291c35dcfb1f19d177927"
server_key_iv: "3ba01fce1777e16b09f90eb6886475fa"
server_key_version: "encryptionkey@0"
I20260811 05:15:41.474830 22487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:15:41.487774 22487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:15:41.490111 22487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:15:41.490239 22487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:15:41.490466 22487 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:15:41.490823 22487 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:15:41.490862 22487 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:41.490885 22487 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:15:41.490897 22487 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:41.496976 22487 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.131:36833
I20260811 05:15:41.497025 22711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.131:36833 every 8 connection(s)
I20260811 05:15:41.497356 22487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
Aug 11 05:15:41 dist-test-slave-f7th krb5kdc[21829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425341, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:15:41.505376 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 22487
I20260811 05:15:41.505514 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance
I20260811 05:15:41.505689 29246 external_mini_cluster.cc:1428] Setting key 259810a37678bbe977e59bdbb73d530d
I20260811 05:15:41.506975 21977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:41.498178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:57261 (local address 127.28.143.190:43443)
0811 05:15:41.498246 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:41.498249 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:41.498444 (+ 195us) server_negotiation.cc:407] Connection header received
0811 05:15:41.499028 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:41.499029 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:41.499042 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:41.499105 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:41.500101 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:41.500665 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:41.501654 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:41.501846 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:41.501896 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:41.504094 (+ 2198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:41.504096 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:41.504096 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:41.504105 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:41.505319 (+ 1214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:41.505839 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:41.505841 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:41.505841 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:41.505924 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:41.506226 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:41.506227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:41.506228 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:41.506424 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:41.506497 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:41.506804 (+ 307us) server_negotiation.cc:299] Negotiation successful
0811 05:15:41.506909 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 05:15:41.507134 22718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:41.498150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.131:57261)
0811 05:15:41.498321 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:41.498361 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:41.498894 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:41.499164 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:41.499168 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:41.499789 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:15:41.500004 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:41.500009 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:41.500749 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:41.500750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:41.501574 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:41.501577 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:41.501908 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:41.501908 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:41.502043 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:41.502536 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:15:41.502545 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:15:41.503972 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:15:41.505401 (+ 1429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:41.505411 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:41.505414 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:41.505722 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:41.506012 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:41.506013 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:41.506013 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:41.506161 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:41.506497 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:15:41.506503 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:15:41.506662 (+ 159us) client_negotiation.cc:769] Sending connection context
0811 05:15:41.506774 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 05:15:41.506926 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":96,"threads_started":1}
I20260811 05:15:41.507586 22712 heartbeater.cc:347] Connected to a master server at 127.28.143.190:43443
I20260811 05:15:41.507680 22712 heartbeater.cc:464] Registering TS with master...
I20260811 05:15:41.507867 22712 heartbeater.cc:511] Master 127.28.143.190:43443 requested a full tablet report, sending...
I20260811 05:15:41.508460 21903 ts_manager.cc:194] Registered new tserver with Master: 4add4ab031844102a37662ac5da7da58 (127.28.143.131:36833)
I20260811 05:15:41.508891 21903 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.131@KRBTEST.COM'} at 127.28.143.131:57261
I20260811 05:15:41.517863 29246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:15:41.525671 21977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:41.519258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40880 (local address 127.28.143.190:43443)
0811 05:15:41.519320 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:41.519322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:41.519502 (+ 180us) server_negotiation.cc:407] Connection header received
0811 05:15:41.519560 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:41.519561 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:41.519571 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:41.519620 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:41.520246 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:41.520798 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:41.521570 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:41.521773 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:41.521832 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:41.523150 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:41.523152 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:41.523153 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:41.523160 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:41.524417 (+ 1257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:41.524852 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:41.524853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:41.524854 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:41.524916 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:41.525066 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:41.525066 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:41.525067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:41.525222 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:41.525301 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:41.525513 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 05:15:41.525597 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 05:15:41.527858 21903 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40880:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:15:41.529659 21903 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:15:41.537057 22737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:41.532597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36242 (local address 127.28.143.130:38185)
0811 05:15:41.533142 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:41.533146 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:41.533156 (+ 10us) server_negotiation.cc:407] Connection header received
0811 05:15:41.533194 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:41.533196 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:41.533212 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:41.533282 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:15:41.533693 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:41.534433 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:41.536019 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:41.536649 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:41.536720 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:41.536790 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:41.536923 (+ 133us) server_negotiation.cc:299] Negotiation successful
0811 05:15:41.536969 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":95,"threads_started":1}
I20260811 05:15:41.537091 22733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:41.532421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:34427 (local address 127.0.0.1:52884)
0811 05:15:41.532841 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:41.532860 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:41.533365 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:41.533648 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:41.533649 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:41.533651 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:15:41.533702 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:41.533704 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:41.534865 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:41.534866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:41.535799 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:41.535802 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:41.536847 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:41.536848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:41.536912 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:41.536916 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:15:41.536965 (+ 49us) client_negotiation.cc:240] Negotiation successful
0811 05:15:41.537007 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":76,"threads_started":1}
I20260811 05:15:41.537182 22743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:41.532779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44672 (local address 127.28.143.131:36833)
0811 05:15:41.532944 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:41.532949 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:41.533030 (+ 81us) server_negotiation.cc:407] Connection header received
0811 05:15:41.533093 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:41.533095 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:41.533112 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:41.533197 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:15:41.533376 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:41.534509 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:41.536001 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:41.536715 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:41.536783 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:41.536829 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:41.537065 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 05:15:41.537108 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":78,"threads_started":1}
I20260811 05:15:41.537315 22735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:41.532679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.131:36833 (local address 127.0.0.1:44672)
0811 05:15:41.532967 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:41.532976 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:41.533026 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:41.533214 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:41.533218 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:41.533220 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:15:41.533301 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:41.533304 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:41.534601 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:41.534602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:41.535913 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:41.535917 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:41.536858 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:41.536859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:41.536925 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:41.536929 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:15:41.536971 (+ 42us) client_negotiation.cc:240] Negotiation successful
0811 05:15:41.537007 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"mutex_wait_us":45,"thread_start_us":76,"threads_started":1}
I20260811 05:15:41.537321 22736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:41.532526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52884 (local address 127.28.143.129:34427)
0811 05:15:41.532799 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:41.532805 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:41.532926 (+ 121us) server_negotiation.cc:407] Connection header received
0811 05:15:41.533462 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:41.533465 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:41.533482 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:41.533557 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:15:41.533790 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:41.534777 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:41.535890 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:41.536675 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:41.536748 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:41.536807 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:41.537198 (+ 391us) server_negotiation.cc:299] Negotiation successful
0811 05:15:41.537242 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":142,"threads_started":1}
I20260811 05:15:41.537868 22741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:41.532412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:38185 (local address 127.0.0.1:36242)
0811 05:15:41.532812 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:41.532859 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:41.532933 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:41.533443 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:41.533444 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:41.533445 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:15:41.533606 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:41.533609 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:41.534533 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:41.534534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:41.535913 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:41.535917 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:41.536720 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:41.536721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:41.536808 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:41.536823 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 05:15:41.536895 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 05:15:41.536959 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":39,"thread_start_us":83,"threads_started":1}
I20260811 05:15:41.538065 22115 tablet_service.cc:1467] Processing CreateTablet for tablet eef54cd740e74f7497ffc880482fa0f1 (DEFAULT_TABLE table=test-table [id=311c17d6e24a476aa6a704d514b377bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:15:41.538072 22594 tablet_service.cc:1467] Processing CreateTablet for tablet eef54cd740e74f7497ffc880482fa0f1 (DEFAULT_TABLE table=test-table [id=311c17d6e24a476aa6a704d514b377bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:15:41.538434 22594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eef54cd740e74f7497ffc880482fa0f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:15:41.538435 22115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eef54cd740e74f7497ffc880482fa0f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:15:41.538617 22350 tablet_service.cc:1467] Processing CreateTablet for tablet eef54cd740e74f7497ffc880482fa0f1 (DEFAULT_TABLE table=test-table [id=311c17d6e24a476aa6a704d514b377bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:15:41.538928 22350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eef54cd740e74f7497ffc880482fa0f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:15:41.541836 22746 tablet_bootstrap.cc:492] T eef54cd740e74f7497ffc880482fa0f1 P 1097935f8aa34158a3bda341d0425928: Bootstrap starting.
I20260811 05:15:41.542296 22748 tablet_bootstrap.cc:492] T eef54cd740e74f7497ffc880482fa0f1 P 40fd4235fcb941ba9da073d48d2599a2: Bootstrap starting.
I20260811 05:15:41.542413 22746 tablet_bootstrap.cc:654] T eef54cd740e74f7497ffc880482fa0f1 P 1097935f8aa34158a3bda341d0425928: Neither blocks nor log segments found. Creating new log.
I20260811 05:15:41.542497 22745 tablet_bootstrap.cc:492] T eef54cd740e74f7497ffc880482fa0f1 P 4add4ab031844102a37662ac5da7da58: Bootstrap starting.
I20260811 05:15:41.542999 22748 tablet_bootstrap.cc:654] T eef54cd740e74f7497ffc880482fa0f1 P 40fd4235fcb941ba9da073d48d2599a2: Neither blocks nor log segments found. Creating new log.
I20260811 05:15:41.543022 22746 log.cc:824] T eef54cd740e74f7497ffc880482fa0f1 P 1097935f8aa34158a3bda341d0425928: Log is configured to *not* fsync() on all Append() calls
I20260811 05:15:41.543095 22745 tablet_bootstrap.cc:654] T eef54cd740e74f7497ffc880482fa0f1 P 4add4ab031844102a37662ac5da7da58: Neither blocks nor log segments found. Creating new log.
I20260811 05:15:41.543576 22748 log.cc:824] T eef54cd740e74f7497ffc880482fa0f1 P 40fd4235fcb941ba9da073d48d2599a2: Log is configured to *not* fsync() on all Append() calls
I20260811 05:15:41.543601 22745 log.cc:824] T eef54cd740e74f7497ffc880482fa0f1 P 4add4ab031844102a37662ac5da7da58: Log is configured to *not* fsync() on all Append() calls
I20260811 05:15:41.544064 22746 tablet_bootstrap.cc:492] T eef54cd740e74f7497ffc880482fa0f1 P 1097935f8aa34158a3bda341d0425928: No bootstrap required, opened a new log
I20260811 05:15:41.544142 22746 ts_tablet_manager.cc:1397] T eef54cd740e74f7497ffc880482fa0f1 P 1097935f8aa34158a3bda341d0425928: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 05:15:41.545287 22748 tablet_bootstrap.cc:492] T eef54cd740e74f7497ffc880482fa0f1 P 40fd4235fcb941ba9da073d48d2599a2: No bootstrap required, opened a new log
I20260811 05:15:41.545372 22748 ts_tablet_manager.cc:1397] T eef54cd740e74f7497ffc880482fa0f1 P 40fd4235fcb941ba9da073d48d2599a2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 05:15:41.545562 22745 tablet_bootstrap.cc:492] T eef54cd740e74f7497ffc880482fa0f1 P 4add4ab031844102a37662ac5da7da58: No bootstrap required, opened a new log
I20260811 05:15:41.545632 22745 ts_tablet_manager.cc:1397] T eef54cd740e74f7497ffc880482fa0f1 P 4add4ab031844102a37662ac5da7da58: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260811 05:15:41.545900 22746 raft_consensus.cc:348] T eef54cd740e74f7497ffc880482fa0f1 P 1097935f8aa34158a3bda341d0425928 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1097935f8aa34158a3bda341d0425928" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 34427 } } peers { permanent_uuid: "4add4ab031844102a37662ac5da7da58" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 36833 } } peers { permanent_uuid: "40fd4235fcb941ba9da073d48d2599a2" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 38185 } }
I20260811 05:15:41.546051 22746 raft_consensus.cc:374] T eef54cd740e74f7497ffc880482fa0f1 P 1097935f8aa34158a3bda341d0425928 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:15:41.546087 22746 raft_consensus.cc:729] T eef54cd740e74f7497ffc880482fa0f1 P 1097935f8aa34158a3bda341d0425928 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1097935f8aa34158a3bda341d0425928, State: Initialized, Role: FOLLOWER
I20260811 05:15:41.546204 22746 consensus_queue.cc:260] T eef54cd740e74f7497ffc880482fa0f1 P 1097935f8aa34158a3bda341d0425928 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1097935f8aa34158a3bda341d0425928" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 34427 } } peers { permanent_uuid: "4add4ab031844102a37662ac5da7da58" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 36833 } } peers { permanent_uuid: "40fd4235fcb941ba9da073d48d2599a2" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 38185 } }
I20260811 05:15:41.546509 22746 ts_tablet_manager.cc:1428] T eef54cd740e74f7497ffc880482fa0f1 P 1097935f8aa34158a3bda341d0425928: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 05:15:41.546518 22226 heartbeater.cc:503] Master 127.28.143.190:43443 was elected leader, sending a full tablet report...
I20260811 05:15:41.547551 22745 raft_consensus.cc:348] T eef54cd740e74f7497ffc880482fa0f1 P 4add4ab031844102a37662ac5da7da58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1097935f8aa34158a3bda341d0425928" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 34427 } } peers { permanent_uuid: "4add4ab031844102a37662ac5da7da58" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 36833 } } peers { permanent_uuid: "40fd4235fcb941ba9da073d48d2599a2" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 38185 } }
I20260811 05:15:41.547551 22748 raft_consensus.cc:348] T eef54cd740e74f7497ffc880482fa0f1 P 40fd4235fcb941ba9da073d48d2599a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1097935f8aa34158a3bda341d0425928" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 34427 } } peers { permanent_uuid: "4add4ab031844102a37662ac5da7da58" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 36833 } } peers { permanent_uuid: "40fd4235fcb941ba9da073d48d2599a2" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 38185 } }
I20260811 05:15:41.547719 22748 raft_consensus.cc:374] T eef54cd740e74f7497ffc880482fa0f1 P 40fd4235fcb941ba9da073d48d2599a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:15:41.547719 22745 raft_consensus.cc:374] T eef54cd740e74f7497ffc880482fa0f1 P 4add4ab031844102a37662ac5da7da58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:15:41.547760 22748 raft_consensus.cc:729] T eef54cd740e74f7497ffc880482fa0f1 P 40fd4235fcb941ba9da073d48d2599a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40fd4235fcb941ba9da073d48d2599a2, State: Initialized, Role: FOLLOWER
I20260811 05:15:41.547760 22745 raft_consensus.cc:729] T eef54cd740e74f7497ffc880482fa0f1 P 4add4ab031844102a37662ac5da7da58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4add4ab031844102a37662ac5da7da58, State: Initialized, Role: FOLLOWER
I20260811 05:15:41.547874 22745 consensus_queue.cc:260] T eef54cd740e74f7497ffc880482fa0f1 P 4add4ab031844102a37662ac5da7da58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1097935f8aa34158a3bda341d0425928" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 34427 } } peers { permanent_uuid: "4add4ab031844102a37662ac5da7da58" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 36833 } } peers { permanent_uuid: "40fd4235fcb941ba9da073d48d2599a2" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 38185 } }
I20260811 05:15:41.547883 22748 consensus_queue.cc:260] T eef54cd740e74f7497ffc880482fa0f1 P 40fd4235fcb941ba9da073d48d2599a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1097935f8aa34158a3bda341d0425928" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 34427 } } peers { permanent_uuid: "4add4ab031844102a37662ac5da7da58" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 36833 } } peers { permanent_uuid: "40fd4235fcb941ba9da073d48d2599a2" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 38185 } }
I20260811 05:15:41.548172 22745 ts_tablet_manager.cc:1428] T eef54cd740e74f7497ffc880482fa0f1 P 4add4ab031844102a37662ac5da7da58: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 05:15:41.548172 22748 ts_tablet_manager.cc:1428] T eef54cd740e74f7497ffc880482fa0f1 P 40fd4235fcb941ba9da073d48d2599a2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 05:15:41.548184 22478 heartbeater.cc:503] Master 127.28.143.190:43443 was elected leader, sending a full tablet report...
I20260811 05:15:41.548270 22712 heartbeater.cc:503] Master 127.28.143.190:43443 was elected leader, sending a full tablet report...
I20260811 05:15:41.551149 22753 raft_consensus.cc:482] T eef54cd740e74f7497ffc880482fa0f1 P 4add4ab031844102a37662ac5da7da58 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:15:41.551245 22753 raft_consensus.cc:504] T eef54cd740e74f7497ffc880482fa0f1 P 4add4ab031844102a37662ac5da7da58 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1097935f8aa34158a3bda341d0425928" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 34427 } } peers { permanent_uuid: "4add4ab031844102a37662ac5da7da58" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 36833 } } peers { permanent_uuid: "40fd4235fcb941ba9da073d48d2599a2" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 38185 } }
I20260811 05:15:41.551678 22753 leader_election.cc:288] T eef54cd740e74f7497ffc880482fa0f1 P 4add4ab031844102a37662ac5da7da58 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1097935f8aa34158a3bda341d0425928 (127.28.143.129:34427), 40fd4235fcb941ba9da073d48d2599a2 (127.28.143.130:38185)
I20260811 05:15:41.555528 22737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:41.552023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:33367 (local address 127.28.143.130:38185)
0811 05:15:41.552079 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:41.552081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:41.552087 (+ 6us) server_negotiation.cc:407] Connection header received
0811 05:15:41.552109 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:41.552110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:41.552117 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:41.552164 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:15:41.552328 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:41.553121 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:41.554531 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:41.555100 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:41.555182 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:41.555193 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:41.555412 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 05:15:41.555457 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 05:15:41.555533 22736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:41.551849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:49093 (local address 127.28.143.129:34427)
0811 05:15:41.551890 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:41.551892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:41.552079 (+ 187us) server_negotiation.cc:407] Connection header received
0811 05:15:41.552179 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:41.552180 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:41.552189 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:41.552236 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:15:41.552359 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:41.553121 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:41.554559 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:41.555185 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:41.555248 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:41.555256 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:41.555435 (+ 179us) server_negotiation.cc:299] Negotiation successful
0811 05:15:41.555475 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 05:15:41.555514 22718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:41.551812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:38185 (local address 127.28.143.131:33367)
0811 05:15:41.551961 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:41.551971 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:41.552013 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:41.552189 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:41.552190 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:41.552192 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:15:41.552262 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:41.552264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:41.553208 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:41.553209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:41.554442 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:41.554445 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:41.555198 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:41.555199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:41.555302 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:41.555308 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 05:15:41.555375 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 05:15:41.555420 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":12}
I20260811 05:15:41.555532 22756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:41.551818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:34427 (local address 127.28.143.131:49093)
0811 05:15:41.551993 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:41.552004 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:41.552106 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:41.552239 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:41.552239 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:41.552241 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:15:41.552298 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:41.552299 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:41.553208 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:41.553209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:41.554442 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:41.554445 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:41.555270 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:41.555271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:41.555331 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:41.555333 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 05:15:41.555427 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 05:15:41.555460 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":73,"thread_start_us":107,"threads_started":1}
I20260811 05:15:41.555895 22146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef54cd740e74f7497ffc880482fa0f1" candidate_uuid: "4add4ab031844102a37662ac5da7da58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1097935f8aa34158a3bda341d0425928" is_pre_election: true
I20260811 05:15:41.556062 22146 raft_consensus.cc:2454] T eef54cd740e74f7497ffc880482fa0f1 P 1097935f8aa34158a3bda341d0425928 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4add4ab031844102a37662ac5da7da58 in term 0.
I20260811 05:15:41.556166 22391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef54cd740e74f7497ffc880482fa0f1" candidate_uuid: "4add4ab031844102a37662ac5da7da58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40fd4235fcb941ba9da073d48d2599a2" is_pre_election: true
I20260811 05:15:41.556285 22515 leader_election.cc:302] T eef54cd740e74f7497ffc880482fa0f1 P 4add4ab031844102a37662ac5da7da58 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1097935f8aa34158a3bda341d0425928, 4add4ab031844102a37662ac5da7da58; no voters:
I20260811 05:15:41.556329 22391 raft_consensus.cc:2454] T eef54cd740e74f7497ffc880482fa0f1 P 40fd4235fcb941ba9da073d48d2599a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4add4ab031844102a37662ac5da7da58 in term 0.
I20260811 05:15:41.556432 22753 raft_consensus.cc:2781] T eef54cd740e74f7497ffc880482fa0f1 P 4add4ab031844102a37662ac5da7da58 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:15:41.556504 22753 raft_consensus.cc:482] T eef54cd740e74f7497ffc880482fa0f1 P 4add4ab031844102a37662ac5da7da58 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:15:41.556529 22753 raft_consensus.cc:3037] T eef54cd740e74f7497ffc880482fa0f1 P 4add4ab031844102a37662ac5da7da58 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:15:41.557353 22753 raft_consensus.cc:504] T eef54cd740e74f7497ffc880482fa0f1 P 4add4ab031844102a37662ac5da7da58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1097935f8aa34158a3bda341d0425928" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 34427 } } peers { permanent_uuid: "4add4ab031844102a37662ac5da7da58" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 36833 } } peers { permanent_uuid: "40fd4235fcb941ba9da073d48d2599a2" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 38185 } }
I20260811 05:15:41.557495 22753 leader_election.cc:288] T eef54cd740e74f7497ffc880482fa0f1 P 4add4ab031844102a37662ac5da7da58 [CANDIDATE]: Term 1 election: Requested vote from peers 1097935f8aa34158a3bda341d0425928 (127.28.143.129:34427), 40fd4235fcb941ba9da073d48d2599a2 (127.28.143.130:38185)
I20260811 05:15:41.557655 22391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef54cd740e74f7497ffc880482fa0f1" candidate_uuid: "4add4ab031844102a37662ac5da7da58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40fd4235fcb941ba9da073d48d2599a2"
I20260811 05:15:41.557678 22146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eef54cd740e74f7497ffc880482fa0f1" candidate_uuid: "4add4ab031844102a37662ac5da7da58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1097935f8aa34158a3bda341d0425928"
I20260811 05:15:41.557749 22146 raft_consensus.cc:3037] T eef54cd740e74f7497ffc880482fa0f1 P 1097935f8aa34158a3bda341d0425928 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:15:41.557749 22391 raft_consensus.cc:3037] T eef54cd740e74f7497ffc880482fa0f1 P 40fd4235fcb941ba9da073d48d2599a2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:15:41.558705 22146 raft_consensus.cc:2454] T eef54cd740e74f7497ffc880482fa0f1 P 1097935f8aa34158a3bda341d0425928 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4add4ab031844102a37662ac5da7da58 in term 1.
I20260811 05:15:41.558705 22391 raft_consensus.cc:2454] T eef54cd740e74f7497ffc880482fa0f1 P 40fd4235fcb941ba9da073d48d2599a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4add4ab031844102a37662ac5da7da58 in term 1.
I20260811 05:15:41.558900 22515 leader_election.cc:302] T eef54cd740e74f7497ffc880482fa0f1 P 4add4ab031844102a37662ac5da7da58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1097935f8aa34158a3bda341d0425928, 4add4ab031844102a37662ac5da7da58; no voters:
I20260811 05:15:41.558988 22753 raft_consensus.cc:2781] T eef54cd740e74f7497ffc880482fa0f1 P 4add4ab031844102a37662ac5da7da58 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:15:41.559149 22753 raft_consensus.cc:686] T eef54cd740e74f7497ffc880482fa0f1 P 4add4ab031844102a37662ac5da7da58 [term 1 LEADER]: Becoming Leader. State: Replica: 4add4ab031844102a37662ac5da7da58, State: Running, Role: LEADER
I20260811 05:15:41.559239 22753 consensus_queue.cc:237] T eef54cd740e74f7497ffc880482fa0f1 P 4add4ab031844102a37662ac5da7da58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1097935f8aa34158a3bda341d0425928" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 34427 } } peers { permanent_uuid: "4add4ab031844102a37662ac5da7da58" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 36833 } } peers { permanent_uuid: "40fd4235fcb941ba9da073d48d2599a2" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 38185 } }
I20260811 05:15:41.560223 21902 catalog_manager.cc:5495] T eef54cd740e74f7497ffc880482fa0f1 P 4add4ab031844102a37662ac5da7da58 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4add4ab031844102a37662ac5da7da58 (127.28.143.131). New cstate: current_term: 1 leader_uuid: "4add4ab031844102a37662ac5da7da58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1097935f8aa34158a3bda341d0425928" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 34427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4add4ab031844102a37662ac5da7da58" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 36833 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40fd4235fcb941ba9da073d48d2599a2" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 38185 } health_report { overall_health: UNKNOWN } } }
I20260811 05:15:41.567870 22743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:41.565265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44686 (local address 127.28.143.131:36833)
0811 05:15:41.565317 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:41.565319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:41.565328 (+ 9us) server_negotiation.cc:407] Connection header received
0811 05:15:41.565362 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:41.565362 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:41.565379 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:41.565450 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:15:41.565677 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:41.566167 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:41.567011 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:41.567201 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:41.567264 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:41.567513 (+ 249us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:15:41.567628 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:15:41.567701 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:41.567789 (+ 88us) server_negotiation.cc:299] Negotiation successful
0811 05:15:41.567808 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 05:15:41.570029 22391 raft_consensus.cc:1261] T eef54cd740e74f7497ffc880482fa0f1 P 40fd4235fcb941ba9da073d48d2599a2 [term 1 FOLLOWER]: Refusing update from remote peer 4add4ab031844102a37662ac5da7da58: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:15:41.570029 22146 raft_consensus.cc:1261] T eef54cd740e74f7497ffc880482fa0f1 P 1097935f8aa34158a3bda341d0425928 [term 1 FOLLOWER]: Refusing update from remote peer 4add4ab031844102a37662ac5da7da58: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:15:41.570451 22758 consensus_queue.cc:1035] T eef54cd740e74f7497ffc880482fa0f1 P 4add4ab031844102a37662ac5da7da58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1097935f8aa34158a3bda341d0425928" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:15:41.570626 22753 consensus_queue.cc:1035] T eef54cd740e74f7497ffc880482fa0f1 P 4add4ab031844102a37662ac5da7da58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40fd4235fcb941ba9da073d48d2599a2" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 38185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:15:41.574074 22769 mvcc.cc:205] Tried to move back new op lower bound from 7317198199067451392 to 7317198199027478528. Current Snapshot: MvccSnapshot[applied={T|T < 7317198199067451392}]
I20260811 05:15:41.574327 22768 mvcc.cc:205] Tried to move back new op lower bound from 7317198199067451392 to 7317198199027478528. Current Snapshot: MvccSnapshot[applied={T|T < 7317198199067451392}]
I20260811 05:15:41.576011 21902 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40880:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:15:41.576107 21902 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40880:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:15:41.576984 21902 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff: Sending DeleteTablet for 3 replicas of tablet eef54cd740e74f7497ffc880482fa0f1
I20260811 05:15:41.577342 22115 tablet_service.cc:1514] Processing DeleteTablet for tablet eef54cd740e74f7497ffc880482fa0f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:15:41 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:52884
I20260811 05:15:41.577342 22594 tablet_service.cc:1514] Processing DeleteTablet for tablet eef54cd740e74f7497ffc880482fa0f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:15:41 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:44672
I20260811 05:15:41.577410 22350 tablet_service.cc:1514] Processing DeleteTablet for tablet eef54cd740e74f7497ffc880482fa0f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:15:41 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:36242
I20260811 05:15:41.577608 22780 tablet_replica.cc:330] T eef54cd740e74f7497ffc880482fa0f1 P 1097935f8aa34158a3bda341d0425928: stopping tablet replica
I20260811 05:15:41.577695 22782 tablet_replica.cc:330] T eef54cd740e74f7497ffc880482fa0f1 P 40fd4235fcb941ba9da073d48d2599a2: stopping tablet replica
I20260811 05:15:41.577723 22780 raft_consensus.cc:2229] T eef54cd740e74f7497ffc880482fa0f1 P 1097935f8aa34158a3bda341d0425928 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:15:41.577745 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 22004
I20260811 05:15:41.614073 22781 tablet_replica.cc:330] T eef54cd740e74f7497ffc880482fa0f1 P 4add4ab031844102a37662ac5da7da58: stopping tablet replica
I20260811 05:15:41.614291 22781 raft_consensus.cc:2229] T eef54cd740e74f7497ffc880482fa0f1 P 4add4ab031844102a37662ac5da7da58 [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:15:41.614516 22781 raft_consensus.cc:2258] T eef54cd740e74f7497ffc880482fa0f1 P 4add4ab031844102a37662ac5da7da58 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:15:41.623822 22781 ts_tablet_manager.cc:1905] T eef54cd740e74f7497ffc880482fa0f1 P 4add4ab031844102a37662ac5da7da58: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:15:41.577790 22782 raft_consensus.cc:2229] T eef54cd740e74f7497ffc880482fa0f1 P 40fd4235fcb941ba9da073d48d2599a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:15:41.636840 22782 raft_consensus.cc:2258] T eef54cd740e74f7497ffc880482fa0f1 P 40fd4235fcb941ba9da073d48d2599a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:15:41.637189 22781 ts_tablet_manager.cc:1918] T eef54cd740e74f7497ffc880482fa0f1 P 4add4ab031844102a37662ac5da7da58: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:15:41.637288 22781 log.cc:1196] T eef54cd740e74f7497ffc880482fa0f1 P 4add4ab031844102a37662ac5da7da58: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/wal/wals/eef54cd740e74f7497ffc880482fa0f1
I20260811 05:15:41.637565 22781 ts_tablet_manager.cc:1939] T eef54cd740e74f7497ffc880482fa0f1 P 4add4ab031844102a37662ac5da7da58: Deleting consensus metadata
I20260811 05:15:41.638105 21884 catalog_manager.cc:4841] TS 4add4ab031844102a37662ac5da7da58 (127.28.143.131:36833): tablet eef54cd740e74f7497ffc880482fa0f1 (table test-table [id=311c17d6e24a476aa6a704d514b377bb]) successfully deleted
I20260811 05:15:41.638378 22782 ts_tablet_manager.cc:1905] T eef54cd740e74f7497ffc880482fa0f1 P 40fd4235fcb941ba9da073d48d2599a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:15:41.640167 22782 ts_tablet_manager.cc:1918] T eef54cd740e74f7497ffc880482fa0f1 P 40fd4235fcb941ba9da073d48d2599a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:15:41.640254 22782 log.cc:1196] T eef54cd740e74f7497ffc880482fa0f1 P 40fd4235fcb941ba9da073d48d2599a2: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/wal/wals/eef54cd740e74f7497ffc880482fa0f1
I20260811 05:15:41.640544 22782 ts_tablet_manager.cc:1939] T eef54cd740e74f7497ffc880482fa0f1 P 40fd4235fcb941ba9da073d48d2599a2: Deleting consensus metadata
I20260811 05:15:41.641103 21880 catalog_manager.cc:4841] TS 40fd4235fcb941ba9da073d48d2599a2 (127.28.143.130:38185): tablet eef54cd740e74f7497ffc880482fa0f1 (table test-table [id=311c17d6e24a476aa6a704d514b377bb]) successfully deleted
W20260811 05:15:41.642833 21884 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.143.129:34427 (error 108)
I20260811 05:15:41.642999 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 22238
I20260811 05:15:41.646309 22741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:41.643352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:34427 (local address 127.0.0.1:52892)
0811 05:15:41.646175 (+ 2823us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:41.646259 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.129:34427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2764}
W20260811 05:15:41.646620 21884 catalog_manager.cc:4584] TS 1097935f8aa34158a3bda341d0425928 (127.28.143.129:34427): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet eef54cd740e74f7497ffc880482fa0f1: Network error: Client connection negotiation failed: client connection to 127.28.143.129:34427: connect: Connection refused (error 111)
I20260811 05:15:41.657689 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 22487
I20260811 05:15:41.697855 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 21855
I20260811 05:15:41.718063 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:43443
--webserver_interface=127.28.143.190
--webserver_port=43551
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:42659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:43443
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:15:41.805022 22784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:15:41.805272 22784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:15:41.805296 22784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:15:41.806598 22784 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:15:41.806658 22784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:15:41.806671 22784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:15:41.806684 22784 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:15:41.806694 22784 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:15:41.808364 22784 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:42659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:43443
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:43443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=43551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:15:41.808661 22784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:15:41.808918 22784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:15:41.819671 22794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:41.819715 22799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:41.819859 22784 server_base.cc:1034] running on GCE node
W20260811 05:15:41.819728 22793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:41.820247 22784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:15:41.820668 22784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:15:41.821861 22784 hybrid_clock.cc:648] HybridClock initialized: now 1786425341821838 us; error 41 us; skew 500 ppm
Aug 11 05:15:41 dist-test-slave-f7th krb5kdc[21829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425341, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:15:41.825634 22784 init.cc:377] Logged in from keytab as kudu/127.28.143.190@KRBTEST.COM (short username kudu)
I20260811 05:15:41.826282 22784 webserver.cc:460] Webserver started at http://127.28.143.190:43551/ using document root <none> and password file <none>
I20260811 05:15:41.826648 22784 fs_manager.cc:359] Metadata directory not provided
I20260811 05:15:41.826730 22784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:15:41.828433 22784 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 05:15:41.829411 22808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:41.829770 22784 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 05:15:41.829892 22784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "b717aaaa6c264b3f9f6dd77dd28788ff"
format_stamp: "Formatted at 2026-08-11 05:15:40 on dist-test-slave-f7th"
server_key: "d628d0324e6406adc48a51a0d75fc9d7"
server_key_iv: "d96516cb84150e99e92cb2dbe92430fb"
server_key_version: "encryptionkey@0"
I20260811 05:15:41.830335 22784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:15:41.874807 22784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:15:41.877233 22784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:15:41.877403 22784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:15:41.881724 22784 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:43443
I20260811 05:15:41.881793 22918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:43443 every 8 connection(s)
I20260811 05:15:41.882197 22784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:15:41.884079 22921 sys_catalog.cc:263] Verifying existing consensus state
I20260811 05:15:41.884788 22921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff: Bootstrap starting.
I20260811 05:15:41.885037 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 22784
I20260811 05:15:41.885397 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.129:34427
--local_ip_for_outbound_sockets=127.28.143.129
--tserver_master_addrs=127.28.143.190:43443
--webserver_port=37341
--webserver_interface=127.28.143.129
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:42659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:15:41.894150 22921 log.cc:824] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff: Log is configured to *not* fsync() on all Append() calls
I20260811 05:15:41.896410 22921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 05:15:41.896775 22921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff: Bootstrap complete.
I20260811 05:15:41.899063 22921 raft_consensus.cc:348] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b717aaaa6c264b3f9f6dd77dd28788ff" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 43443 } }
I20260811 05:15:41.899323 22921 raft_consensus.cc:729] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b717aaaa6c264b3f9f6dd77dd28788ff, State: Initialized, Role: FOLLOWER
I20260811 05:15:41.899488 22921 consensus_queue.cc:260] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b717aaaa6c264b3f9f6dd77dd28788ff" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 43443 } }
I20260811 05:15:41.899560 22921 raft_consensus.cc:388] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:15:41.899616 22921 raft_consensus.cc:482] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:15:41.899667 22921 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [term 1 FOLLOWER]: Advancing to term 2
I20260811 05:15:41.900990 22921 raft_consensus.cc:504] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b717aaaa6c264b3f9f6dd77dd28788ff" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 43443 } }
I20260811 05:15:41.901108 22921 leader_election.cc:302] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b717aaaa6c264b3f9f6dd77dd28788ff; no voters:
I20260811 05:15:41.901304 22921 leader_election.cc:288] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 05:15:41.901358 22927 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [term 2 FOLLOWER]: Leader election won for term 2
I20260811 05:15:41.901489 22927 raft_consensus.cc:686] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [term 2 LEADER]: Becoming Leader. State: Replica: b717aaaa6c264b3f9f6dd77dd28788ff, State: Running, Role: LEADER
I20260811 05:15:41.901623 22921 sys_catalog.cc:564] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:15:41.901598 22927 consensus_queue.cc:237] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b717aaaa6c264b3f9f6dd77dd28788ff" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 43443 } }
I20260811 05:15:41.902202 22932 sys_catalog.cc:455] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader b717aaaa6c264b3f9f6dd77dd28788ff. Latest consensus state: current_term: 2 leader_uuid: "b717aaaa6c264b3f9f6dd77dd28788ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b717aaaa6c264b3f9f6dd77dd28788ff" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 43443 } } }
I20260811 05:15:41.902309 22932 sys_catalog.cc:458] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [sys.catalog]: This master's current role is: LEADER
I20260811 05:15:41.902482 22931 sys_catalog.cc:455] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b717aaaa6c264b3f9f6dd77dd28788ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b717aaaa6c264b3f9f6dd77dd28788ff" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 43443 } } }
I20260811 05:15:41.902556 22931 sys_catalog.cc:458] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [sys.catalog]: This master's current role is: LEADER
I20260811 05:15:41.902664 22942 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:15:41.903622 22942 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:15:41.903744 22942 catalog_manager.cc:1259] Loaded cluster ID: cf5ae5b924c64c05879742bfcfaf8b3a
I20260811 05:15:41.904017 22942 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:15:41.904310 22942 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 05:15:41.908980 22942 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 05:15:41.912691 22942 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:15:41.912981 22942 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff: Loaded TSK: 0
I20260811 05:15:41.913218 22942 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 05:15:41.981122 22925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:15:41.981364 22925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:15:41.981392 22925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:15:41.982800 22925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:15:41.982864 22925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:15:41.982894 22925 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.129
I20260811 05:15:41.984576 22925 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:42659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.129:34427
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.143.129
--webserver_port=37341
--tserver_master_addrs=127.28.143.190:43443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22925
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.129
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:15:41.984918 22925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:15:41.985215 22925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:15:41.990820 22960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:41.990892 22962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:41.990984 22925 server_base.cc:1034] running on GCE node
W20260811 05:15:41.991179 22965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:41.991427 22925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:15:41.991791 22925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:15:41.992985 22925 hybrid_clock.cc:648] HybridClock initialized: now 1786425341992896 us; error 100 us; skew 500 ppm
Aug 11 05:15:41 dist-test-slave-f7th krb5kdc[21829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425341, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:15:41.996665 22925 init.cc:377] Logged in from keytab as kudu/127.28.143.129@KRBTEST.COM (short username kudu)
I20260811 05:15:41.997392 22925 webserver.cc:460] Webserver started at http://127.28.143.129:37341/ using document root <none> and password file <none>
I20260811 05:15:41.997723 22925 fs_manager.cc:359] Metadata directory not provided
I20260811 05:15:41.997817 22925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:15:41.999966 22925 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 05:15:42.001385 22979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:42.001605 22925 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 05:15:42.001735 22925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/wal
uuid: "1097935f8aa34158a3bda341d0425928"
format_stamp: "Formatted at 2026-08-11 05:15:41 on dist-test-slave-f7th"
server_key: "4353b587e41a7fff4f97414272035c06"
server_key_iv: "85a4f5b7dc193e53a34bd02c50395da0"
server_key_version: "encryptionkey@0"
I20260811 05:15:42.002254 22925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:15:42.021297 22925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:15:42.024468 22925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:15:42.024735 22925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:15:42.025071 22925 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:15:42.025786 22991 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 05:15:42.027451 22925 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 05:15:42.027531 22925 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260811 05:15:42.027571 22925 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 05:15:42.028594 22925 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 05:15:42.028685 22925 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.001s
I20260811 05:15:42.028779 22991 tablet_bootstrap.cc:492] T eef54cd740e74f7497ffc880482fa0f1 P 1097935f8aa34158a3bda341d0425928: Bootstrap starting.
I20260811 05:15:42.036734 22925 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.129:34427
I20260811 05:15:42.037293 22925 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
I20260811 05:15:42.039528 22991 log.cc:824] T eef54cd740e74f7497ffc880482fa0f1 P 1097935f8aa34158a3bda341d0425928: Log is configured to *not* fsync() on all Append() calls
I20260811 05:15:42.041146 23171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.129:34427 every 8 connection(s)
I20260811 05:15:42.042470 22991 tablet_bootstrap.cc:492] T eef54cd740e74f7497ffc880482fa0f1 P 1097935f8aa34158a3bda341d0425928: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 05:15:42.042735 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 22925
I20260811 05:15:42.042805 22991 tablet_bootstrap.cc:492] T eef54cd740e74f7497ffc880482fa0f1 P 1097935f8aa34158a3bda341d0425928: Bootstrap complete.
I20260811 05:15:42.043054 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.130:38185
--local_ip_for_outbound_sockets=127.28.143.130
--tserver_master_addrs=127.28.143.190:43443
--webserver_port=34815
--webserver_interface=127.28.143.130
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:42659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:15:42.043949 22991 ts_tablet_manager.cc:1397] T eef54cd740e74f7497ffc880482fa0f1 P 1097935f8aa34158a3bda341d0425928: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.011s
I20260811 05:15:42.045492 22991 raft_consensus.cc:348] T eef54cd740e74f7497ffc880482fa0f1 P 1097935f8aa34158a3bda341d0425928 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1097935f8aa34158a3bda341d0425928" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 34427 } } peers { permanent_uuid: "4add4ab031844102a37662ac5da7da58" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 36833 } } peers { permanent_uuid: "40fd4235fcb941ba9da073d48d2599a2" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 38185 } }
I20260811 05:15:42.045848 22991 raft_consensus.cc:729] T eef54cd740e74f7497ffc880482fa0f1 P 1097935f8aa34158a3bda341d0425928 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1097935f8aa34158a3bda341d0425928, State: Initialized, Role: FOLLOWER
I20260811 05:15:42.046007 22991 consensus_queue.cc:260] T eef54cd740e74f7497ffc880482fa0f1 P 1097935f8aa34158a3bda341d0425928 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1097935f8aa34158a3bda341d0425928" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 34427 } } peers { permanent_uuid: "4add4ab031844102a37662ac5da7da58" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 36833 } } peers { permanent_uuid: "40fd4235fcb941ba9da073d48d2599a2" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 38185 } }
I20260811 05:15:42.046339 22991 ts_tablet_manager.cc:1428] T eef54cd740e74f7497ffc880482fa0f1 P 1097935f8aa34158a3bda341d0425928: Time spent starting tablet: real 0.002s user 0.006s sys 0.000s
Aug 11 05:15:42 dist-test-slave-f7th krb5kdc[21829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425341, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:15:42.054661 23180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:42.038331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:60491 (local address 127.28.143.190:43443)
0811 05:15:42.040874 (+ 2543us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:42.040886 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:42.040940 (+ 54us) server_negotiation.cc:407] Connection header received
0811 05:15:42.041097 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:42.041104 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:42.041169 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:42.041323 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:42.042186 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:42.043896 (+ 1710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:42.045597 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:42.046795 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:42.046948 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:42.049935 (+ 2987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:42.049941 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:42.049942 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:42.049957 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:42.052236 (+ 2279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:42.052954 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:42.052960 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:42.052960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:42.053107 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:42.053477 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:42.053478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:42.053479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:42.053798 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:42.053905 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:42.054294 (+ 389us) server_negotiation.cc:299] Negotiation successful
0811 05:15:42.054448 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2477,"thread_start_us":114,"threads_started":1}
I20260811 05:15:42.054673 23174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:42.038209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.129:60491)
0811 05:15:42.039830 (+ 1621us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:42.039885 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:42.040537 (+ 652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:42.041498 (+ 961us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:42.041502 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:42.041833 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:15:42.042045 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:42.042051 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:42.044099 (+ 2048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:42.044101 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:42.045462 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:42.045466 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:42.046974 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:42.046976 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:42.047130 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:42.047750 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:15:42.047761 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:15:42.049783 (+ 2022us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:15:42.052375 (+ 2592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:42.052387 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:42.052390 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:42.052803 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:42.053197 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:42.053198 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:42.053199 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:42.053384 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:42.053893 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:15:42.053901 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:15:42.054091 (+ 190us) client_negotiation.cc:769] Sending connection context
0811 05:15:42.054219 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 05:15:42.054436 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1570,"thread_start_us":115,"threads_started":1}
I20260811 05:15:42.056371 23172 heartbeater.cc:347] Connected to a master server at 127.28.143.190:43443
I20260811 05:15:42.056491 23172 heartbeater.cc:464] Registering TS with master...
I20260811 05:15:42.056734 23172 heartbeater.cc:511] Master 127.28.143.190:43443 requested a full tablet report, sending...
I20260811 05:15:42.057654 22847 ts_manager.cc:194] Registered new tserver with Master: 1097935f8aa34158a3bda341d0425928 (127.28.143.129:34427)
I20260811 05:15:42.058944 22847 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.129@KRBTEST.COM'} at 127.28.143.129:60491
I20260811 05:15:42.060148 23172 heartbeater.cc:503] Master 127.28.143.190:43443 was elected leader, sending a full tablet report...
I20260811 05:15:42.065716 23185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:42.058408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:34427 (local address 127.0.0.1:52894)
0811 05:15:42.058928 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:42.058968 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:42.059054 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:42.062204 (+ 3150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:42.062209 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:42.062213 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:15:42.062297 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:42.062299 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:42.063164 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:42.063166 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:42.064535 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:42.064538 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:42.065354 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:42.065355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:42.065466 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:42.065483 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 05:15:42.065561 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 05:15:42.065623 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":137,"threads_started":1}
I20260811 05:15:42.065722 23187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:42.058712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52894 (local address 127.28.143.129:34427)
0811 05:15:42.062028 (+ 3316us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:42.062032 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:42.062043 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:15:42.062077 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:42.062079 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:42.062093 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:42.062170 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:15:42.062380 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:42.063081 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:42.064690 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:42.065263 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:42.065337 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:42.065381 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:42.065619 (+ 238us) server_negotiation.cc:299] Negotiation successful
0811 05:15:42.065664 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3281,"thread_start_us":91,"threads_started":1}
I20260811 05:15:42.066188 23064 tablet_service.cc:1514] Processing DeleteTablet for tablet eef54cd740e74f7497ffc880482fa0f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:15:41 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:52894
I20260811 05:15:42.066742 23192 tablet_replica.cc:330] stopping tablet replica
I20260811 05:15:42.066871 23192 raft_consensus.cc:2229] T eef54cd740e74f7497ffc880482fa0f1 P 1097935f8aa34158a3bda341d0425928 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:15:42.066943 23192 raft_consensus.cc:2258] T eef54cd740e74f7497ffc880482fa0f1 P 1097935f8aa34158a3bda341d0425928 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:15:42.067301 23192 ts_tablet_manager.cc:1905] T eef54cd740e74f7497ffc880482fa0f1 P 1097935f8aa34158a3bda341d0425928: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:15:42.069901 23192 ts_tablet_manager.cc:1918] T eef54cd740e74f7497ffc880482fa0f1 P 1097935f8aa34158a3bda341d0425928: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:15:42.069991 23192 log.cc:1196] T eef54cd740e74f7497ffc880482fa0f1 P 1097935f8aa34158a3bda341d0425928: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/wal/wals/eef54cd740e74f7497ffc880482fa0f1
I20260811 05:15:42.070207 23192 ts_tablet_manager.cc:1939] T eef54cd740e74f7497ffc880482fa0f1 P 1097935f8aa34158a3bda341d0425928: Deleting consensus metadata
I20260811 05:15:42.070775 22826 catalog_manager.cc:4841] TS 1097935f8aa34158a3bda341d0425928 (127.28.143.129:34427): tablet eef54cd740e74f7497ffc880482fa0f1 (table test-table [id=311c17d6e24a476aa6a704d514b377bb]) successfully deleted
W20260811 05:15:42.134629 23182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:15:42.134908 23182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:15:42.134943 23182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:15:42.136132 23182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:15:42.136169 23182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:15:42.136196 23182 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.130
I20260811 05:15:42.137739 23182 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:42659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.130:38185
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.143.130
--webserver_port=34815
--tserver_master_addrs=127.28.143.190:43443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23182
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.130
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:15:42.137960 23182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:15:42.138157 23182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 05:15:42.143476 23182 server_base.cc:1034] running on GCE node
W20260811 05:15:42.143472 23199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:42.143450 23204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:42.143445 23207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:42.143842 23182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:15:42.144091 23182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:15:42.145246 23182 hybrid_clock.cc:648] HybridClock initialized: now 1786425342145231 us; error 29 us; skew 500 ppm
Aug 11 05:15:42 dist-test-slave-f7th krb5kdc[21829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425342, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:15:42.147804 23182 init.cc:377] Logged in from keytab as kudu/127.28.143.130@KRBTEST.COM (short username kudu)
I20260811 05:15:42.148293 23182 webserver.cc:460] Webserver started at http://127.28.143.130:34815/ using document root <none> and password file <none>
I20260811 05:15:42.148527 23182 fs_manager.cc:359] Metadata directory not provided
I20260811 05:15:42.148599 23182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:15:42.149981 23182 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:15:42.150729 23216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:42.151021 23182 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:15:42.151135 23182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/wal
uuid: "40fd4235fcb941ba9da073d48d2599a2"
format_stamp: "Formatted at 2026-08-11 05:15:41 on dist-test-slave-f7th"
server_key: "944e62cded5923f47f98069b91f3ced2"
server_key_iv: "6d4db8d3d27771a48e72c7fbf78d0ba4"
server_key_version: "encryptionkey@0"
I20260811 05:15:42.151518 23182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:15:42.173792 23182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:15:42.176043 23182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:15:42.176196 23182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:15:42.176450 23182 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:15:42.176811 23182 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:15:42.176849 23182 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:42.176882 23182 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:15:42.176899 23182 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:42.182611 23182 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.130:38185
I20260811 05:15:42.182680 23390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.130:38185 every 8 connection(s)
I20260811 05:15:42.183030 23182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
Aug 11 05:15:42 dist-test-slave-f7th krb5kdc[21829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425342, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:15:42.191152 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 23182
I20260811 05:15:42.191433 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.131:36833
--local_ip_for_outbound_sockets=127.28.143.131
--tserver_master_addrs=127.28.143.190:43443
--webserver_port=39033
--webserver_interface=127.28.143.131
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:42659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:15:42.192970 23180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:42.183810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:54525 (local address 127.28.143.190:43443)
0811 05:15:42.183879 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:42.183881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:42.184106 (+ 225us) server_negotiation.cc:407] Connection header received
0811 05:15:42.184596 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:42.184597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:42.184611 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:42.184704 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:42.185716 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:42.186231 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:42.187305 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:42.187507 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:42.187588 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:42.189941 (+ 2353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:42.189943 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:42.189943 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:42.189950 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:42.191405 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:42.191923 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:42.191924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:42.191925 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:42.191989 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:42.192294 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:42.192294 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:42.192295 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:42.192412 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:42.192462 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:42.192808 (+ 346us) server_negotiation.cc:299] Negotiation successful
0811 05:15:42.192899 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 05:15:42.193212 23395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:42.183796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.130:54525)
0811 05:15:42.183974 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:42.184015 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:42.184499 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:42.184772 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:42.184775 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:42.185406 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:15:42.185620 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:42.185625 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:42.186354 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:42.186355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:42.187213 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:42.187216 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:42.187609 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:42.187610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:42.187726 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:42.188302 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:15:42.188311 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:15:42.189806 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:15:42.191500 (+ 1694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:42.191510 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:42.191513 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:42.191819 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:42.192077 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:42.192078 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:42.192078 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:42.192233 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:42.192466 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:15:42.192472 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:15:42.192680 (+ 208us) client_negotiation.cc:769] Sending connection context
0811 05:15:42.192794 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 05:15:42.192973 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":88,"threads_started":1}
I20260811 05:15:42.193728 23393 heartbeater.cc:347] Connected to a master server at 127.28.143.190:43443
I20260811 05:15:42.193853 23393 heartbeater.cc:464] Registering TS with master...
I20260811 05:15:42.194084 23393 heartbeater.cc:511] Master 127.28.143.190:43443 requested a full tablet report, sending...
I20260811 05:15:42.194571 22847 ts_manager.cc:194] Registered new tserver with Master: 40fd4235fcb941ba9da073d48d2599a2 (127.28.143.130:38185)
I20260811 05:15:42.194972 22847 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.130@KRBTEST.COM'} at 127.28.143.130:54525
W20260811 05:15:42.282256 23398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:15:42.282477 23398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:15:42.282500 23398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:15:42.283669 23398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:15:42.283715 23398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:15:42.283741 23398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.131
I20260811 05:15:42.285247 23398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:42659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.131:36833
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.143.131
--webserver_port=39033
--tserver_master_addrs=127.28.143.190:43443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.131
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:15:42.285512 23398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:15:42.285749 23398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:15:42.290970 23408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:42.290954 23409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:42.290956 23411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:42.291227 23398 server_base.cc:1034] running on GCE node
I20260811 05:15:42.291435 23398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:15:42.291697 23398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:15:42.292868 23398 hybrid_clock.cc:648] HybridClock initialized: now 1786425342292851 us; error 29 us; skew 500 ppm
Aug 11 05:15:42 dist-test-slave-f7th krb5kdc[21829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425342, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:15:42.295423 23398 init.cc:377] Logged in from keytab as kudu/127.28.143.131@KRBTEST.COM (short username kudu)
I20260811 05:15:42.295892 23398 webserver.cc:460] Webserver started at http://127.28.143.131:39033/ using document root <none> and password file <none>
I20260811 05:15:42.296108 23398 fs_manager.cc:359] Metadata directory not provided
I20260811 05:15:42.296176 23398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:15:42.297636 23398 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:15:42.298219 23426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:42.298367 23398 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 05:15:42.298452 23398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/wal
uuid: "4add4ab031844102a37662ac5da7da58"
format_stamp: "Formatted at 2026-08-11 05:15:41 on dist-test-slave-f7th"
server_key: "0fb23a895c5291c35dcfb1f19d177927"
server_key_iv: "3ba01fce1777e16b09f90eb6886475fa"
server_key_version: "encryptionkey@0"
I20260811 05:15:42.298787 23398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:15:42.315306 23398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:15:42.317545 23398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:15:42.317703 23398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:15:42.317952 23398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:15:42.318326 23398 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:15:42.318367 23398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:42.318398 23398 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:15:42.318415 23398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:42.324693 23398 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.131:36833
I20260811 05:15:42.324733 23610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.131:36833 every 8 connection(s)
I20260811 05:15:42.325107 23398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
I20260811 05:15:42.328447 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 23398
Aug 11 05:15:42 dist-test-slave-f7th krb5kdc[21829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425342, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:15:42.335354 23180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:42.325881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:60569 (local address 127.28.143.190:43443)
0811 05:15:42.325993 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:42.325996 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:42.326194 (+ 198us) server_negotiation.cc:407] Connection header received
0811 05:15:42.326757 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:42.326758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:42.326773 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:42.326826 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:42.327815 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:42.328314 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:42.329602 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:42.329806 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:42.329871 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:42.332098 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:42.332100 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:42.332100 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:42.332108 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:42.333769 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:42.334252 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:42.334253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:42.334253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:42.334302 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:42.334619 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:42.334620 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:42.334620 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:42.334767 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:42.334827 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:42.335163 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 05:15:42.335252 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 05:15:42.335537 23621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:42.328955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40888 (local address 127.28.143.190:43443)
0811 05:15:42.329076 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:42.329078 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:42.329161 (+ 83us) server_negotiation.cc:407] Connection header received
0811 05:15:42.329203 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:42.329203 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:42.329215 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:42.329281 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:42.329858 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:42.330329 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:42.331056 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:42.331255 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:42.331307 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:42.332651 (+ 1344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:42.332653 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:42.332653 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:42.332660 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:42.334348 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:42.334677 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:42.334678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:42.334679 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:42.334730 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:42.334948 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:42.334949 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:42.334949 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:42.335052 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:42.335220 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:42.335291 (+ 71us) server_negotiation.cc:299] Negotiation successful
0811 05:15:42.335425 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":55,"threads_started":1}
I20260811 05:15:42.335881 23614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:42.325848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.131:60569)
0811 05:15:42.326061 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:42.326101 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:42.326645 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:42.326911 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:42.326915 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:42.327515 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:15:42.327711 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:42.327715 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:42.328401 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:42.328402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:42.329530 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:42.329532 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:42.329876 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:42.329877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:42.330009 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:42.330529 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:15:42.330539 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:15:42.332005 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:15:42.333849 (+ 1844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:42.333857 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:42.333861 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:42.334174 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:42.334369 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:42.334370 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:42.334370 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:42.334546 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:42.334826 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:15:42.334833 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:15:42.335005 (+ 172us) client_negotiation.cc:769] Sending connection context
0811 05:15:42.335465 (+ 460us) client_negotiation.cc:240] Negotiation successful
0811 05:15:42.335647 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":97,"threads_started":1}
I20260811 05:15:42.336297 23612 heartbeater.cc:347] Connected to a master server at 127.28.143.190:43443
I20260811 05:15:42.336402 23612 heartbeater.cc:464] Registering TS with master...
I20260811 05:15:42.336591 23612 heartbeater.cc:511] Master 127.28.143.190:43443 requested a full tablet report, sending...
I20260811 05:15:42.336987 22844 ts_manager.cc:194] Registered new tserver with Master: 4add4ab031844102a37662ac5da7da58 (127.28.143.131:36833)
I20260811 05:15:42.337357 22844 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.131@KRBTEST.COM'} at 127.28.143.131:60569
I20260811 05:15:42.347456 29246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:15:42.350358 23621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:42.347830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40894 (local address 127.28.143.190:43443)
0811 05:15:42.347913 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:42.347915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:42.348082 (+ 167us) server_negotiation.cc:407] Connection header received
0811 05:15:42.348127 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:42.348128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:42.348167 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:42.348232 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:15:42.348370 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:42.348946 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:42.349681 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:42.349888 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:42.349942 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:42.350072 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:15:42.350176 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:15:42.350233 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:42.350249 (+ 16us) server_negotiation.cc:299] Negotiation successful
0811 05:15:42.350268 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 05:15:42.350664 22844 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:40894:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:15:42.352315 22844 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:15:42.355067 23064 tablet_service.cc:1467] Processing CreateTablet for tablet 384993a5669b48769dea0a0b9824a78e (DEFAULT_TABLE table=test-table [id=7993df55efbf4e2b94f03de6d39fca10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:15:42.355304 23064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 384993a5669b48769dea0a0b9824a78e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:15:42.356993 22991 tablet_bootstrap.cc:492] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928: Bootstrap starting.
I20260811 05:15:42.357347 22991 tablet_bootstrap.cc:654] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928: Neither blocks nor log segments found. Creating new log.
I20260811 05:15:42.358079 22991 tablet_bootstrap.cc:492] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928: No bootstrap required, opened a new log
I20260811 05:15:42.358144 22991 ts_tablet_manager.cc:1397] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:15:42.358273 22991 raft_consensus.cc:348] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fd4235fcb941ba9da073d48d2599a2" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 38185 } } peers { permanent_uuid: "4add4ab031844102a37662ac5da7da58" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 36833 } } peers { permanent_uuid: "1097935f8aa34158a3bda341d0425928" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 34427 } }
I20260811 05:15:42.358349 22991 raft_consensus.cc:374] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:15:42.358410 22991 raft_consensus.cc:729] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1097935f8aa34158a3bda341d0425928, State: Initialized, Role: FOLLOWER
I20260811 05:15:42.358492 23185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:42.354428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.131:36833 (local address 127.0.0.1:44700)
0811 05:15:42.354476 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:42.354489 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:42.354533 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:42.355005 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:42.355006 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:42.355008 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:15:42.355053 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:42.355055 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:42.356359 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:42.356360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:42.357234 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:42.357239 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:42.358263 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:42.358264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:42.358330 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:42.358336 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 05:15:42.358367 (+ 31us) client_negotiation.cc:240] Negotiation successful
0811 05:15:42.358424 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260811 05:15:42.358477 22991 consensus_queue.cc:260] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fd4235fcb941ba9da073d48d2599a2" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 38185 } } peers { permanent_uuid: "4add4ab031844102a37662ac5da7da58" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 36833 } } peers { permanent_uuid: "1097935f8aa34158a3bda341d0425928" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 34427 } }
I20260811 05:15:42.358580 22991 ts_tablet_manager.cc:1428] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 05:15:42.358911 23624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:42.354660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44700 (local address 127.28.143.131:36833)
0811 05:15:42.354844 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:42.354849 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:42.354861 (+ 12us) server_negotiation.cc:407] Connection header received
0811 05:15:42.354903 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:42.354905 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:42.354923 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:42.355010 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:15:42.355106 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:42.356231 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:42.357427 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:42.358157 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:42.358676 (+ 519us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:42.358751 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:42.358782 (+ 31us) server_negotiation.cc:299] Negotiation successful
0811 05:15:42.358826 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":99,"threads_started":1}
I20260811 05:15:42.359578 23501 tablet_service.cc:1467] Processing CreateTablet for tablet 384993a5669b48769dea0a0b9824a78e (DEFAULT_TABLE table=test-table [id=7993df55efbf4e2b94f03de6d39fca10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:15:42.359937 23501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 384993a5669b48769dea0a0b9824a78e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:15:42.360544 23623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:42.354443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:38185 (local address 127.0.0.1:36246)
0811 05:15:42.354715 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:42.354726 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:42.354779 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:42.357227 (+ 2448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:42.357229 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:42.357231 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:15:42.357282 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:42.357284 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:42.358489 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:42.358491 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:42.359461 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:42.359464 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:42.360288 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:42.360289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:42.360362 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:42.360366 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:15:42.360422 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 05:15:42.360468 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":100,"thread_start_us":98,"threads_started":1}
I20260811 05:15:42.360562 23625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:42.354688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36246 (local address 127.28.143.130:38185)
0811 05:15:42.357063 (+ 2375us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:42.357069 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:42.357081 (+ 12us) server_negotiation.cc:407] Connection header received
0811 05:15:42.357127 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:42.357129 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:42.357147 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:42.357229 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:15:42.357337 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:42.358280 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:42.359543 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:42.360211 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:42.360280 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:42.360336 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:42.360439 (+ 103us) server_negotiation.cc:299] Negotiation successful
0811 05:15:42.360481 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2328,"thread_start_us":98,"threads_started":1}
I20260811 05:15:42.361197 23288 tablet_service.cc:1467] Processing CreateTablet for tablet 384993a5669b48769dea0a0b9824a78e (DEFAULT_TABLE table=test-table [id=7993df55efbf4e2b94f03de6d39fca10]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:15:42.361526 23288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 384993a5669b48769dea0a0b9824a78e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:15:42.364349 23628 tablet_bootstrap.cc:492] T 384993a5669b48769dea0a0b9824a78e P 40fd4235fcb941ba9da073d48d2599a2: Bootstrap starting.
I20260811 05:15:42.364459 23627 tablet_bootstrap.cc:492] T 384993a5669b48769dea0a0b9824a78e P 4add4ab031844102a37662ac5da7da58: Bootstrap starting.
I20260811 05:15:42.365059 23628 tablet_bootstrap.cc:654] T 384993a5669b48769dea0a0b9824a78e P 40fd4235fcb941ba9da073d48d2599a2: Neither blocks nor log segments found. Creating new log.
I20260811 05:15:42.365062 23627 tablet_bootstrap.cc:654] T 384993a5669b48769dea0a0b9824a78e P 4add4ab031844102a37662ac5da7da58: Neither blocks nor log segments found. Creating new log.
I20260811 05:15:42.365527 23627 log.cc:824] T 384993a5669b48769dea0a0b9824a78e P 4add4ab031844102a37662ac5da7da58: Log is configured to *not* fsync() on all Append() calls
I20260811 05:15:42.365528 23628 log.cc:824] T 384993a5669b48769dea0a0b9824a78e P 40fd4235fcb941ba9da073d48d2599a2: Log is configured to *not* fsync() on all Append() calls
I20260811 05:15:42.366276 23628 tablet_bootstrap.cc:492] T 384993a5669b48769dea0a0b9824a78e P 40fd4235fcb941ba9da073d48d2599a2: No bootstrap required, opened a new log
I20260811 05:15:42.366302 23627 tablet_bootstrap.cc:492] T 384993a5669b48769dea0a0b9824a78e P 4add4ab031844102a37662ac5da7da58: No bootstrap required, opened a new log
I20260811 05:15:42.366357 23628 ts_tablet_manager.cc:1397] T 384993a5669b48769dea0a0b9824a78e P 40fd4235fcb941ba9da073d48d2599a2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 05:15:42.366376 23627 ts_tablet_manager.cc:1397] T 384993a5669b48769dea0a0b9824a78e P 4add4ab031844102a37662ac5da7da58: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 05:15:42.368005 23627 raft_consensus.cc:348] T 384993a5669b48769dea0a0b9824a78e P 4add4ab031844102a37662ac5da7da58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fd4235fcb941ba9da073d48d2599a2" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 38185 } } peers { permanent_uuid: "4add4ab031844102a37662ac5da7da58" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 36833 } } peers { permanent_uuid: "1097935f8aa34158a3bda341d0425928" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 34427 } }
I20260811 05:15:42.367908 23628 raft_consensus.cc:348] T 384993a5669b48769dea0a0b9824a78e P 40fd4235fcb941ba9da073d48d2599a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fd4235fcb941ba9da073d48d2599a2" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 38185 } } peers { permanent_uuid: "4add4ab031844102a37662ac5da7da58" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 36833 } } peers { permanent_uuid: "1097935f8aa34158a3bda341d0425928" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 34427 } }
I20260811 05:15:42.368175 23627 raft_consensus.cc:374] T 384993a5669b48769dea0a0b9824a78e P 4add4ab031844102a37662ac5da7da58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:15:42.368211 23627 raft_consensus.cc:729] T 384993a5669b48769dea0a0b9824a78e P 4add4ab031844102a37662ac5da7da58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4add4ab031844102a37662ac5da7da58, State: Initialized, Role: FOLLOWER
I20260811 05:15:42.368201 23628 raft_consensus.cc:374] T 384993a5669b48769dea0a0b9824a78e P 40fd4235fcb941ba9da073d48d2599a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:15:42.368258 23628 raft_consensus.cc:729] T 384993a5669b48769dea0a0b9824a78e P 40fd4235fcb941ba9da073d48d2599a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40fd4235fcb941ba9da073d48d2599a2, State: Initialized, Role: FOLLOWER
I20260811 05:15:42.368378 23627 consensus_queue.cc:260] T 384993a5669b48769dea0a0b9824a78e P 4add4ab031844102a37662ac5da7da58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fd4235fcb941ba9da073d48d2599a2" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 38185 } } peers { permanent_uuid: "4add4ab031844102a37662ac5da7da58" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 36833 } } peers { permanent_uuid: "1097935f8aa34158a3bda341d0425928" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 34427 } }
I20260811 05:15:42.368391 23628 consensus_queue.cc:260] T 384993a5669b48769dea0a0b9824a78e P 40fd4235fcb941ba9da073d48d2599a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fd4235fcb941ba9da073d48d2599a2" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 38185 } } peers { permanent_uuid: "4add4ab031844102a37662ac5da7da58" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 36833 } } peers { permanent_uuid: "1097935f8aa34158a3bda341d0425928" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 34427 } }
I20260811 05:15:42.368738 23612 heartbeater.cc:503] Master 127.28.143.190:43443 was elected leader, sending a full tablet report...
I20260811 05:15:42.368793 23628 ts_tablet_manager.cc:1428] T 384993a5669b48769dea0a0b9824a78e P 40fd4235fcb941ba9da073d48d2599a2: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260811 05:15:42.368821 23393 heartbeater.cc:503] Master 127.28.143.190:43443 was elected leader, sending a full tablet report...
I20260811 05:15:42.368726 23627 ts_tablet_manager.cc:1428] T 384993a5669b48769dea0a0b9824a78e P 4add4ab031844102a37662ac5da7da58: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
W20260811 05:15:42.433643 23394 tablet.cc:2369] T 384993a5669b48769dea0a0b9824a78e P 40fd4235fcb941ba9da073d48d2599a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:15:42.443887 23632 raft_consensus.cc:482] T 384993a5669b48769dea0a0b9824a78e P 4add4ab031844102a37662ac5da7da58 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:15:42.444034 23632 raft_consensus.cc:504] T 384993a5669b48769dea0a0b9824a78e P 4add4ab031844102a37662ac5da7da58 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fd4235fcb941ba9da073d48d2599a2" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 38185 } } peers { permanent_uuid: "4add4ab031844102a37662ac5da7da58" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 36833 } } peers { permanent_uuid: "1097935f8aa34158a3bda341d0425928" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 34427 } }
I20260811 05:15:42.444433 23632 leader_election.cc:288] T 384993a5669b48769dea0a0b9824a78e P 4add4ab031844102a37662ac5da7da58 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 40fd4235fcb941ba9da073d48d2599a2 (127.28.143.130:38185), 1097935f8aa34158a3bda341d0425928 (127.28.143.129:34427)
I20260811 05:15:42.448348 23187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:42.444722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:41123 (local address 127.28.143.129:34427)
0811 05:15:42.444764 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:42.444766 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:42.444980 (+ 214us) server_negotiation.cc:407] Connection header received
0811 05:15:42.445019 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:42.445020 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:42.445028 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:42.445069 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:15:42.445188 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:42.445863 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:42.447385 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:42.447999 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:42.448061 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:42.448067 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:42.448243 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 05:15:42.448283 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 05:15:42.448345 23625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:42.444729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:52329 (local address 127.28.143.130:38185)
0811 05:15:42.444774 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:42.444776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:42.444788 (+ 12us) server_negotiation.cc:407] Connection header received
0811 05:15:42.444845 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:42.444846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:42.444856 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:42.444912 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:15:42.445090 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:42.445836 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:42.447397 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:42.447960 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:42.448040 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:42.448053 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:42.448220 (+ 167us) server_negotiation.cc:299] Negotiation successful
0811 05:15:42.448271 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 05:15:42.448367 23635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:42.444685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:34427 (local address 127.28.143.131:41123)
0811 05:15:42.444899 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:42.444910 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:42.444966 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:42.445084 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:42.445085 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:42.445086 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:15:42.445136 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:42.445137 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:42.445957 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:42.445957 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:42.447274 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:42.447277 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:42.448106 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:42.448108 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:42.448171 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:42.448175 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:15:42.448221 (+ 46us) client_negotiation.cc:240] Negotiation successful
0811 05:15:42.448264 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":40,"thread_start_us":131,"threads_started":1}
I20260811 05:15:42.448500 23614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:42.444658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:38185 (local address 127.28.143.131:52329)
0811 05:15:42.444719 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:42.444730 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:42.444776 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:42.444934 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:42.444935 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:42.444937 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:15:42.445012 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:42.445015 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:42.445944 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:42.445945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:42.447274 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:42.447277 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:42.448041 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:42.448043 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:42.448137 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:42.448141 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:15:42.448201 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 05:15:42.448246 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":64}
I20260811 05:15:42.448845 23318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "384993a5669b48769dea0a0b9824a78e" candidate_uuid: "4add4ab031844102a37662ac5da7da58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40fd4235fcb941ba9da073d48d2599a2" is_pre_election: true
I20260811 05:15:42.448913 23084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "384993a5669b48769dea0a0b9824a78e" candidate_uuid: "4add4ab031844102a37662ac5da7da58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1097935f8aa34158a3bda341d0425928" is_pre_election: true
I20260811 05:15:42.449007 23318 raft_consensus.cc:2454] T 384993a5669b48769dea0a0b9824a78e P 40fd4235fcb941ba9da073d48d2599a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4add4ab031844102a37662ac5da7da58 in term 0.
I20260811 05:15:42.449074 23084 raft_consensus.cc:2454] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4add4ab031844102a37662ac5da7da58 in term 0.
I20260811 05:15:42.449245 23427 leader_election.cc:302] T 384993a5669b48769dea0a0b9824a78e P 4add4ab031844102a37662ac5da7da58 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40fd4235fcb941ba9da073d48d2599a2, 4add4ab031844102a37662ac5da7da58; no voters:
I20260811 05:15:42.449432 23632 raft_consensus.cc:2781] T 384993a5669b48769dea0a0b9824a78e P 4add4ab031844102a37662ac5da7da58 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:15:42.449503 23632 raft_consensus.cc:482] T 384993a5669b48769dea0a0b9824a78e P 4add4ab031844102a37662ac5da7da58 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:15:42.449520 23632 raft_consensus.cc:3037] T 384993a5669b48769dea0a0b9824a78e P 4add4ab031844102a37662ac5da7da58 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:15:42.450217 23632 raft_consensus.cc:504] T 384993a5669b48769dea0a0b9824a78e P 4add4ab031844102a37662ac5da7da58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fd4235fcb941ba9da073d48d2599a2" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 38185 } } peers { permanent_uuid: "4add4ab031844102a37662ac5da7da58" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 36833 } } peers { permanent_uuid: "1097935f8aa34158a3bda341d0425928" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 34427 } }
I20260811 05:15:42.450340 23632 leader_election.cc:288] T 384993a5669b48769dea0a0b9824a78e P 4add4ab031844102a37662ac5da7da58 [CANDIDATE]: Term 1 election: Requested vote from peers 40fd4235fcb941ba9da073d48d2599a2 (127.28.143.130:38185), 1097935f8aa34158a3bda341d0425928 (127.28.143.129:34427)
I20260811 05:15:42.450536 23084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "384993a5669b48769dea0a0b9824a78e" candidate_uuid: "4add4ab031844102a37662ac5da7da58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1097935f8aa34158a3bda341d0425928"
I20260811 05:15:42.450567 23318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "384993a5669b48769dea0a0b9824a78e" candidate_uuid: "4add4ab031844102a37662ac5da7da58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40fd4235fcb941ba9da073d48d2599a2"
I20260811 05:15:42.450621 23084 raft_consensus.cc:3037] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:15:42.450649 23318 raft_consensus.cc:3037] T 384993a5669b48769dea0a0b9824a78e P 40fd4235fcb941ba9da073d48d2599a2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:15:42.451311 23084 raft_consensus.cc:2454] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4add4ab031844102a37662ac5da7da58 in term 1.
I20260811 05:15:42.451454 23318 raft_consensus.cc:2454] T 384993a5669b48769dea0a0b9824a78e P 40fd4235fcb941ba9da073d48d2599a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4add4ab031844102a37662ac5da7da58 in term 1.
I20260811 05:15:42.451501 23431 leader_election.cc:302] T 384993a5669b48769dea0a0b9824a78e P 4add4ab031844102a37662ac5da7da58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1097935f8aa34158a3bda341d0425928, 4add4ab031844102a37662ac5da7da58; no voters:
I20260811 05:15:42.451623 23632 raft_consensus.cc:2781] T 384993a5669b48769dea0a0b9824a78e P 4add4ab031844102a37662ac5da7da58 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:15:42.451776 23632 raft_consensus.cc:686] T 384993a5669b48769dea0a0b9824a78e P 4add4ab031844102a37662ac5da7da58 [term 1 LEADER]: Becoming Leader. State: Replica: 4add4ab031844102a37662ac5da7da58, State: Running, Role: LEADER
I20260811 05:15:42.451869 23632 consensus_queue.cc:237] T 384993a5669b48769dea0a0b9824a78e P 4add4ab031844102a37662ac5da7da58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fd4235fcb941ba9da073d48d2599a2" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 38185 } } peers { permanent_uuid: "4add4ab031844102a37662ac5da7da58" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 36833 } } peers { permanent_uuid: "1097935f8aa34158a3bda341d0425928" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 34427 } }
I20260811 05:15:42.452787 22847 catalog_manager.cc:5495] T 384993a5669b48769dea0a0b9824a78e P 4add4ab031844102a37662ac5da7da58 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4add4ab031844102a37662ac5da7da58 (127.28.143.131). New cstate: current_term: 1 leader_uuid: "4add4ab031844102a37662ac5da7da58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fd4235fcb941ba9da073d48d2599a2" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 38185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4add4ab031844102a37662ac5da7da58" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 36833 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1097935f8aa34158a3bda341d0425928" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 34427 } health_report { overall_health: UNKNOWN } } }
I20260811 05:15:42.476464 23624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:42.474112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44712 (local address 127.28.143.131:36833)
0811 05:15:42.474180 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:42.474182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:42.474189 (+ 7us) server_negotiation.cc:407] Connection header received
0811 05:15:42.474224 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:42.474225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:42.474243 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:42.474305 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:15:42.474434 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:42.474914 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:42.475695 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:42.475893 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:42.475952 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:42.476102 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:15:42.476216 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:15:42.476289 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:42.476327 (+ 38us) server_negotiation.cc:299] Negotiation successful
0811 05:15:42.476349 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 05:15:42.478482 23084 raft_consensus.cc:1261] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [term 1 FOLLOWER]: Refusing update from remote peer 4add4ab031844102a37662ac5da7da58: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:15:42.478803 23318 raft_consensus.cc:1261] T 384993a5669b48769dea0a0b9824a78e P 40fd4235fcb941ba9da073d48d2599a2 [term 1 FOLLOWER]: Refusing update from remote peer 4add4ab031844102a37662ac5da7da58: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:15:42.478853 23637 consensus_queue.cc:1035] T 384993a5669b48769dea0a0b9824a78e P 4add4ab031844102a37662ac5da7da58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1097935f8aa34158a3bda341d0425928" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 34427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:15:42.479131 23637 consensus_queue.cc:1035] T 384993a5669b48769dea0a0b9824a78e P 4add4ab031844102a37662ac5da7da58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40fd4235fcb941ba9da073d48d2599a2" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 38185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:15:42.481525 23642 mvcc.cc:205] Tried to move back new op lower bound from 7317198202788982784 to 7317198202683596800. Current Snapshot: MvccSnapshot[applied={T|T < 7317198202788982784}]
I20260811 05:15:42.482501 23644 mvcc.cc:205] Tried to move back new op lower bound from 7317198202788982784 to 7317198202683596800. Current Snapshot: MvccSnapshot[applied={T|T < 7317198202788982784 or (T in {7317198202788982784})}]
I20260811 05:15:42.483712 22844 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:40894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:15:42.483826 22844 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:40894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:15:42.484732 22844 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff: Sending DeleteTablet for 3 replicas of tablet 384993a5669b48769dea0a0b9824a78e
I20260811 05:15:42.485102 23288 tablet_service.cc:1514] Processing DeleteTablet for tablet 384993a5669b48769dea0a0b9824a78e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:15:42 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:36246
I20260811 05:15:42.485155 23064 tablet_service.cc:1514] Processing DeleteTablet for tablet 384993a5669b48769dea0a0b9824a78e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:15:42 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:52894
I20260811 05:15:42.485267 23192 tablet_replica.cc:330] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928: stopping tablet replica
I20260811 05:15:42.485334 23192 raft_consensus.cc:2229] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:15:42.485397 23192 raft_consensus.cc:2258] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:15:42.485389 23663 tablet_replica.cc:330] T 384993a5669b48769dea0a0b9824a78e P 40fd4235fcb941ba9da073d48d2599a2: stopping tablet replica
I20260811 05:15:42.485452 23501 tablet_service.cc:1514] Processing DeleteTablet for tablet 384993a5669b48769dea0a0b9824a78e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:15:42 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:44700
I20260811 05:15:42.485505 23663 raft_consensus.cc:2229] T 384993a5669b48769dea0a0b9824a78e P 40fd4235fcb941ba9da073d48d2599a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:15:42.485621 23663 raft_consensus.cc:2258] T 384993a5669b48769dea0a0b9824a78e P 40fd4235fcb941ba9da073d48d2599a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:15:42.485697 23665 tablet_replica.cc:330] T 384993a5669b48769dea0a0b9824a78e P 4add4ab031844102a37662ac5da7da58: stopping tablet replica
I20260811 05:15:42.485791 23665 raft_consensus.cc:2229] T 384993a5669b48769dea0a0b9824a78e P 4add4ab031844102a37662ac5da7da58 [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:15:42.485916 23665 raft_consensus.cc:2258] T 384993a5669b48769dea0a0b9824a78e P 4add4ab031844102a37662ac5da7da58 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:15:42.486110 23663 ts_tablet_manager.cc:1905] T 384993a5669b48769dea0a0b9824a78e P 40fd4235fcb941ba9da073d48d2599a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:15:42.486120 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 22925
I20260811 05:15:42.488178 23663 ts_tablet_manager.cc:1918] T 384993a5669b48769dea0a0b9824a78e P 40fd4235fcb941ba9da073d48d2599a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:15:42.488282 23663 log.cc:1196] T 384993a5669b48769dea0a0b9824a78e P 40fd4235fcb941ba9da073d48d2599a2: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/wal/wals/384993a5669b48769dea0a0b9824a78e
I20260811 05:15:42.488696 23663 ts_tablet_manager.cc:1939] T 384993a5669b48769dea0a0b9824a78e P 40fd4235fcb941ba9da073d48d2599a2: Deleting consensus metadata
I20260811 05:15:42.489280 22814 catalog_manager.cc:4841] TS 40fd4235fcb941ba9da073d48d2599a2 (127.28.143.130:38185): tablet 384993a5669b48769dea0a0b9824a78e (table test-table [id=7993df55efbf4e2b94f03de6d39fca10]) successfully deleted
I20260811 05:15:42.510571 23665 ts_tablet_manager.cc:1905] T 384993a5669b48769dea0a0b9824a78e P 4add4ab031844102a37662ac5da7da58: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:15:42.512723 23665 ts_tablet_manager.cc:1918] T 384993a5669b48769dea0a0b9824a78e P 4add4ab031844102a37662ac5da7da58: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:15:42.512837 23665 log.cc:1196] T 384993a5669b48769dea0a0b9824a78e P 4add4ab031844102a37662ac5da7da58: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/wal/wals/384993a5669b48769dea0a0b9824a78e
I20260811 05:15:42.513152 23665 ts_tablet_manager.cc:1939] T 384993a5669b48769dea0a0b9824a78e P 4add4ab031844102a37662ac5da7da58: Deleting consensus metadata
I20260811 05:15:42.513723 22826 catalog_manager.cc:4841] TS 4add4ab031844102a37662ac5da7da58 (127.28.143.131:36833): tablet 384993a5669b48769dea0a0b9824a78e (table test-table [id=7993df55efbf4e2b94f03de6d39fca10]) successfully deleted
W20260811 05:15:42.514648 22826 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.143.129:34427 (error 108)
I20260811 05:15:42.515177 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 23182
I20260811 05:15:42.515256 23623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:42.515092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:34427 (local address 127.0.0.1:52900)
0811 05:15:42.515165 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:42.515219 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.129:34427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 05:15:42.520596 22826 catalog_manager.cc:4584] TS 1097935f8aa34158a3bda341d0425928 (127.28.143.129:34427): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 384993a5669b48769dea0a0b9824a78e: Network error: Client connection negotiation failed: client connection to 127.28.143.129:34427: connect: Connection refused (error 111)
I20260811 05:15:42.541054 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 23398
I20260811 05:15:42.549372 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 22784
I20260811 05:15:42.567728 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:43443
--webserver_interface=127.28.143.190
--webserver_port=43551
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:42659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:43443
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:15:42.655648 23667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:15:42.655875 23667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:15:42.655900 23667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:15:42.657140 23667 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:15:42.657186 23667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:15:42.657198 23667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:15:42.657209 23667 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:15:42.657222 23667 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:15:42.658761 23667 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:42659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:43443
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:43443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=43551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:15:42.659032 23667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:15:42.659276 23667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:15:42.664297 23673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:42.664300 23672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:42.664453 23667 server_base.cc:1034] running on GCE node
W20260811 05:15:42.664354 23677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:42.664918 23667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:15:42.665254 23667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:15:42.666414 23667 hybrid_clock.cc:648] HybridClock initialized: now 1786425342666393 us; error 37 us; skew 500 ppm
Aug 11 05:15:42 dist-test-slave-f7th krb5kdc[21829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425342, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:15:42.669143 23667 init.cc:377] Logged in from keytab as kudu/127.28.143.190@KRBTEST.COM (short username kudu)
I20260811 05:15:42.669652 23667 webserver.cc:460] Webserver started at http://127.28.143.190:43551/ using document root <none> and password file <none>
I20260811 05:15:42.669904 23667 fs_manager.cc:359] Metadata directory not provided
I20260811 05:15:42.669970 23667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:15:42.671445 23667 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:15:42.672240 23689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:42.672497 23667 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:15:42.672592 23667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "b717aaaa6c264b3f9f6dd77dd28788ff"
format_stamp: "Formatted at 2026-08-11 05:15:40 on dist-test-slave-f7th"
server_key: "d628d0324e6406adc48a51a0d75fc9d7"
server_key_iv: "d96516cb84150e99e92cb2dbe92430fb"
server_key_version: "encryptionkey@0"
I20260811 05:15:42.673009 23667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:15:42.684113 23667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:15:42.686396 23667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:15:42.686550 23667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:15:42.691051 23667 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:43443
I20260811 05:15:42.691089 23772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:43443 every 8 connection(s)
I20260811 05:15:42.691479 23667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:15:42.693097 23774 sys_catalog.cc:263] Verifying existing consensus state
I20260811 05:15:42.693833 23774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff: Bootstrap starting.
I20260811 05:15:42.693990 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 23667
I20260811 05:15:42.694312 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.129:34427
--local_ip_for_outbound_sockets=127.28.143.129
--tserver_master_addrs=127.28.143.190:43443
--webserver_port=37341
--webserver_interface=127.28.143.129
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:42659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:15:42.703361 23774 log.cc:824] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff: Log is configured to *not* fsync() on all Append() calls
I20260811 05:15:42.705971 23774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 05:15:42.706288 23774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff: Bootstrap complete.
I20260811 05:15:42.708523 23774 raft_consensus.cc:348] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b717aaaa6c264b3f9f6dd77dd28788ff" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 43443 } }
I20260811 05:15:42.708820 23774 raft_consensus.cc:729] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: b717aaaa6c264b3f9f6dd77dd28788ff, State: Initialized, Role: FOLLOWER
I20260811 05:15:42.709012 23774 consensus_queue.cc:260] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b717aaaa6c264b3f9f6dd77dd28788ff" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 43443 } }
I20260811 05:15:42.709093 23774 raft_consensus.cc:388] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:15:42.709141 23774 raft_consensus.cc:482] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:15:42.709192 23774 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [term 2 FOLLOWER]: Advancing to term 3
I20260811 05:15:42.710243 23774 raft_consensus.cc:504] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b717aaaa6c264b3f9f6dd77dd28788ff" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 43443 } }
I20260811 05:15:42.710341 23774 leader_election.cc:302] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b717aaaa6c264b3f9f6dd77dd28788ff; no voters:
I20260811 05:15:42.710534 23774 leader_election.cc:288] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 05:15:42.710629 23779 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [term 3 FOLLOWER]: Leader election won for term 3
I20260811 05:15:42.710805 23779 raft_consensus.cc:686] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [term 3 LEADER]: Becoming Leader. State: Replica: b717aaaa6c264b3f9f6dd77dd28788ff, State: Running, Role: LEADER
I20260811 05:15:42.710819 23774 sys_catalog.cc:564] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:15:42.711074 23779 consensus_queue.cc:237] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b717aaaa6c264b3f9f6dd77dd28788ff" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 43443 } }
I20260811 05:15:42.711407 23782 sys_catalog.cc:455] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader b717aaaa6c264b3f9f6dd77dd28788ff. Latest consensus state: current_term: 3 leader_uuid: "b717aaaa6c264b3f9f6dd77dd28788ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b717aaaa6c264b3f9f6dd77dd28788ff" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 43443 } } }
I20260811 05:15:42.711485 23782 sys_catalog.cc:458] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [sys.catalog]: This master's current role is: LEADER
I20260811 05:15:42.711634 23780 sys_catalog.cc:455] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "b717aaaa6c264b3f9f6dd77dd28788ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b717aaaa6c264b3f9f6dd77dd28788ff" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 43443 } } }
I20260811 05:15:42.711688 23780 sys_catalog.cc:458] T 00000000000000000000000000000000 P b717aaaa6c264b3f9f6dd77dd28788ff [sys.catalog]: This master's current role is: LEADER
I20260811 05:15:42.711787 23794 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:15:42.712641 23794 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:15:42.712757 23794 catalog_manager.cc:1259] Loaded cluster ID: cf5ae5b924c64c05879742bfcfaf8b3a
I20260811 05:15:42.712790 23794 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 05:15:42.713037 23794 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fe3c35199ad google::LogMessage::Fail() at ??:0
@ 0x7fe3c351c88a google::LogMessage::SendToLog() at ??:0
@ 0x7fe3c35194de google::LogMessage::Flush() at ??:0
@ 0x7fe3c351aa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fe3cb43a0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fe3cb4438b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fe3c53e70d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fe3c53dc954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fe3c62406db start_thread at ??:0
@ 0x7fe3c201761f clone at ??:0
W20260811 05:15:42.790832 23776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:15:42.791080 23776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:15:42.791118 23776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:15:42.792414 23776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:15:42.792470 23776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:15:42.792507 23776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.129
I20260811 05:15:42.794209 23776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:42659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.129:34427
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.143.129
--webserver_port=37341
--tserver_master_addrs=127.28.143.190:43443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.129
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:15:42.794507 23776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:15:42.794749 23776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:15:42.800148 23836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:42.800225 23776 server_base.cc:1034] running on GCE node
W20260811 05:15:42.800165 23838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:42.800427 23834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:42.800750 23776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:15:42.801069 23776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:15:42.802258 23776 hybrid_clock.cc:648] HybridClock initialized: now 1786425342802245 us; error 31 us; skew 500 ppm
Aug 11 05:15:42 dist-test-slave-f7th krb5kdc[21829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425342, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:15:42.805809 23776 init.cc:377] Logged in from keytab as kudu/127.28.143.129@KRBTEST.COM (short username kudu)
I20260811 05:15:42.806330 23776 webserver.cc:460] Webserver started at http://127.28.143.129:37341/ using document root <none> and password file <none>
I20260811 05:15:42.806566 23776 fs_manager.cc:359] Metadata directory not provided
I20260811 05:15:42.806619 23776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:15:42.808158 23776 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:15:42.809085 23856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:42.809293 23776 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:15:42.809424 23776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/wal
uuid: "1097935f8aa34158a3bda341d0425928"
format_stamp: "Formatted at 2026-08-11 05:15:41 on dist-test-slave-f7th"
server_key: "4353b587e41a7fff4f97414272035c06"
server_key_iv: "85a4f5b7dc193e53a34bd02c50395da0"
server_key_version: "encryptionkey@0"
I20260811 05:15:42.809793 23776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:15:42.819140 23776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:15:42.821473 23776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:15:42.821625 23776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:15:42.821857 23776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:15:42.822288 23868 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 05:15:42.824321 23776 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 05:15:42.824388 23776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260811 05:15:42.824438 23776 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 05:15:42.825706 23776 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 05:15:42.825767 23776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260811 05:15:42.826257 23868 tablet_bootstrap.cc:492] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928: Bootstrap starting.
I20260811 05:15:42.833611 23776 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.129:34427
I20260811 05:15:42.834215 23776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
I20260811 05:15:42.834689 24031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.129:34427 every 8 connection(s)
I20260811 05:15:42.837471 23868 log.cc:824] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928: Log is configured to *not* fsync() on all Append() calls
I20260811 05:15:42.839811 23868 tablet_bootstrap.cc:492] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 05:15:42.840083 23868 tablet_bootstrap.cc:492] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928: Bootstrap complete.
I20260811 05:15:42.840852 23868 ts_tablet_manager.cc:1397] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.012s
I20260811 05:15:42.841145 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 23776
I20260811 05:15:42.841431 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.130:38185
--local_ip_for_outbound_sockets=127.28.143.130
--tserver_master_addrs=127.28.143.190:43443
--webserver_port=34815
--webserver_interface=127.28.143.130
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:42659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:15:42.841958 23868 raft_consensus.cc:348] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fd4235fcb941ba9da073d48d2599a2" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 38185 } } peers { permanent_uuid: "4add4ab031844102a37662ac5da7da58" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 36833 } } peers { permanent_uuid: "1097935f8aa34158a3bda341d0425928" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 34427 } }
I20260811 05:15:42.842181 23868 raft_consensus.cc:729] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1097935f8aa34158a3bda341d0425928, State: Initialized, Role: FOLLOWER
I20260811 05:15:42.842290 23868 consensus_queue.cc:260] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fd4235fcb941ba9da073d48d2599a2" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 38185 } } peers { permanent_uuid: "4add4ab031844102a37662ac5da7da58" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 36833 } } peers { permanent_uuid: "1097935f8aa34158a3bda341d0425928" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 34427 } }
I20260811 05:15:42.842566 23868 ts_tablet_manager.cc:1428] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
W20260811 05:15:42.919333 24035 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:15:42.834997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.129:48053)
0811 05:15:42.835289 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:42.835341 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:42.835978 (+ 637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:42.919064 (+ 83086us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":94,"threads_started":1}
W20260811 05:15:42.919624 24032 heartbeater.cc:644] Failed to heartbeat to 127.28.143.190:43443 (0 consecutive failures): Network error: Failed to ping master at 127.28.143.190:43443: Client connection negotiation failed: client connection to 127.28.143.190:43443: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 05:15:42.920047 24035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:42.919913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.129:36201)
0811 05:15:42.919972 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:42.920022 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260811 05:15:42.920399 24035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:42.920305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.129:34653)
0811 05:15:42.920351 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:42.920380 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260811 05:15:42.920523 24032 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 05:15:42.933557 24038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:15:42.933825 24038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:15:42.933854 24038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:15:42.935354 24038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:15:42.935429 24038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:15:42.935460 24038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.130
I20260811 05:15:42.937296 24038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:42659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.130:38185
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.143.130
--webserver_port=34815
--tserver_master_addrs=127.28.143.190:43443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.130
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:15:42.937565 24038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:15:42.937819 24038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:15:42.943697 24050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:42.943702 24048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:42.943697 24047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:42.943733 24038 server_base.cc:1034] running on GCE node
I20260811 05:15:42.944316 24038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:15:42.944602 24038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:15:42.945809 24038 hybrid_clock.cc:648] HybridClock initialized: now 1786425342945782 us; error 39 us; skew 500 ppm
Aug 11 05:15:42 dist-test-slave-f7th krb5kdc[21829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425342, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:15:42.948589 24038 init.cc:377] Logged in from keytab as kudu/127.28.143.130@KRBTEST.COM (short username kudu)
I20260811 05:15:42.949092 24038 webserver.cc:460] Webserver started at http://127.28.143.130:34815/ using document root <none> and password file <none>
I20260811 05:15:42.949309 24038 fs_manager.cc:359] Metadata directory not provided
I20260811 05:15:42.949360 24038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:15:42.950757 24038 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:15:42.951349 24061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:42.951517 24038 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:42.951614 24038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/wal
uuid: "40fd4235fcb941ba9da073d48d2599a2"
format_stamp: "Formatted at 2026-08-11 05:15:41 on dist-test-slave-f7th"
server_key: "944e62cded5923f47f98069b91f3ced2"
server_key_iv: "6d4db8d3d27771a48e72c7fbf78d0ba4"
server_key_version: "encryptionkey@0"
I20260811 05:15:42.951984 24038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:15:42.968724 24038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:15:42.971340 24038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:15:42.971516 24038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:15:42.971779 24038 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:15:42.972112 24038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:15:42.972148 24038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:42.972180 24038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:15:42.972198 24038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:42.978467 24038 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.130:38185
I20260811 05:15:42.978546 24251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.130:38185 every 8 connection(s)
I20260811 05:15:42.978878 24038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
I20260811 05:15:42.979995 24259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:42.979563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.130:39219)
0811 05:15:42.979750 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:42.979842 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":78,"threads_started":1}
W20260811 05:15:42.980450 24256 heartbeater.cc:644] Failed to heartbeat to 127.28.143.190:43443 (0 consecutive failures): Network error: Failed to ping master at 127.28.143.190:43443: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
I20260811 05:15:42.980780 24259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:42.980698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.130:53701)
0811 05:15:42.980732 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:42.980761 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260811 05:15:42.981139 24259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:42.981039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.130:48879)
0811 05:15:42.981103 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:42.981125 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 05:15:42.981267 24256 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 05:15:42.988755 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 24038
I20260811 05:15:42.989071 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.131:36833
--local_ip_for_outbound_sockets=127.28.143.131
--tserver_master_addrs=127.28.143.190:43443
--webserver_port=39033
--webserver_interface=127.28.143.131
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:42659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:15:43.075680 24260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:15:43.075920 24260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:15:43.075947 24260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:15:43.077255 24260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:15:43.077310 24260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:15:43.077339 24260 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.131
I20260811 05:15:43.078931 24260 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:42659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.131:36833
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.143.131
--webserver_port=39033
--tserver_master_addrs=127.28.143.190:43443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24260
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.131
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:15:43.079205 24260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:15:43.079442 24260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:15:43.084816 24272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:43.084817 24274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:43.084817 24277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:43.084848 24260 server_base.cc:1034] running on GCE node
I20260811 05:15:43.085652 24260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:15:43.085884 24260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:15:43.087020 24260 hybrid_clock.cc:648] HybridClock initialized: now 1786425343087008 us; error 31 us; skew 500 ppm
Aug 11 05:15:43 dist-test-slave-f7th krb5kdc[21829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425343, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:15:43.089782 24260 init.cc:377] Logged in from keytab as kudu/127.28.143.131@KRBTEST.COM (short username kudu)
I20260811 05:15:43.090273 24260 webserver.cc:460] Webserver started at http://127.28.143.131:39033/ using document root <none> and password file <none>
I20260811 05:15:43.090514 24260 fs_manager.cc:359] Metadata directory not provided
I20260811 05:15:43.090566 24260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:15:43.091995 24260 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:15:43.092733 24287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:43.092938 24260 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:15:43.093067 24260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/wal
uuid: "4add4ab031844102a37662ac5da7da58"
format_stamp: "Formatted at 2026-08-11 05:15:41 on dist-test-slave-f7th"
server_key: "0fb23a895c5291c35dcfb1f19d177927"
server_key_iv: "3ba01fce1777e16b09f90eb6886475fa"
server_key_version: "encryptionkey@0"
I20260811 05:15:43.093427 24260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:15:43.103197 24260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:15:43.105664 24260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:15:43.105823 24260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:15:43.106220 24260 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:15:43.106624 24260 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:15:43.106673 24260 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:43.106706 24260 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:15:43.106724 24260 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:43.112507 24260 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.131:36833
I20260811 05:15:43.112571 24467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.131:36833 every 8 connection(s)
I20260811 05:15:43.113001 24260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
I20260811 05:15:43.114121 24472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:43.113684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.131:37439)
0811 05:15:43.113871 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:43.113969 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":80,"threads_started":1}
W20260811 05:15:43.114562 24468 heartbeater.cc:644] Failed to heartbeat to 127.28.143.190:43443 (0 consecutive failures): Network error: Failed to ping master at 127.28.143.190:43443: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
I20260811 05:15:43.114929 24472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:43.114755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.131:59623)
0811 05:15:43.114874 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:43.114906 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260811 05:15:43.115242 24472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:43.115184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.131:41555)
0811 05:15:43.115206 (+ 22us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:43.115228 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260811 05:15:43.115329 24468 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 05:15:43.115583 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 24260
I20260811 05:15:43.921271 24474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:43.920984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.129:46089)
0811 05:15:43.921143 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:43.921194 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":81,"threads_started":1}
I20260811 05:15:43.981932 24477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:43.981652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.130:43911)
0811 05:15:43.981830 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:43.981873 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":77,"threads_started":1}
I20260811 05:15:44.116014 24483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:44.115718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.131:35703)
0811 05:15:44.115906 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:44.115951 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":66,"threads_started":1}
I20260811 05:15:44.636313 24484 raft_consensus.cc:482] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:15:44.636456 24484 raft_consensus.cc:504] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fd4235fcb941ba9da073d48d2599a2" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 38185 } } peers { permanent_uuid: "4add4ab031844102a37662ac5da7da58" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 36833 } } peers { permanent_uuid: "1097935f8aa34158a3bda341d0425928" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 34427 } }
I20260811 05:15:44.636854 24484 leader_election.cc:288] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 40fd4235fcb941ba9da073d48d2599a2 (127.28.143.130:38185), 4add4ab031844102a37662ac5da7da58 (127.28.143.131:36833)
Aug 11 05:15:44 dist-test-slave-f7th krb5kdc[21829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425342, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for kudu/127.28.143.131@KRBTEST.COM
Aug 11 05:15:44 dist-test-slave-f7th krb5kdc[21829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425342, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for kudu/127.28.143.130@KRBTEST.COM
I20260811 05:15:44.647342 24490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:44.637174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:48125 (local address 127.28.143.130:38185)
0811 05:15:44.637343 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:44.637354 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:44.637373 (+ 19us) server_negotiation.cc:407] Connection header received
0811 05:15:44.637832 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:44.637834 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:44.637889 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:44.638016 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:44.638813 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:44.639766 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:44.640909 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:44.641317 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:44.641422 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:44.644128 (+ 2706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:44.644132 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:44.644132 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:44.644146 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:44.645839 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:44.646158 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:44.646161 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:44.646162 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:44.646245 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:44.646419 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:44.646420 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:44.646421 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:44.646693 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:44.646766 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:44.647082 (+ 316us) server_negotiation.cc:299] Negotiation successful
0811 05:15:44.647228 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":74,"threads_started":1}
I20260811 05:15:44.647357 24489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:44.637159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:44623 (local address 127.28.143.131:36833)
0811 05:15:44.637341 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:44.637352 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:44.637373 (+ 21us) server_negotiation.cc:407] Connection header received
0811 05:15:44.637832 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:44.637834 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:44.637878 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:44.638010 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:44.638831 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:44.639765 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:44.640916 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:44.641322 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:44.641422 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:44.643712 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:44.643717 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:44.643718 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:44.643731 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:44.645577 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:44.645965 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:44.645967 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:44.645968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:44.646070 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:44.646310 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:44.646311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:44.646312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:44.646634 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:44.646726 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:44.647142 (+ 416us) server_negotiation.cc:299] Negotiation successful
0811 05:15:44.647262 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":87,"threads_started":1}
I20260811 05:15:44.647346 24488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:44.637087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.131:36833 (local address 127.28.143.129:44623)
0811 05:15:44.637196 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:44.637206 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:44.637264 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:44.638082 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:44.638085 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:44.638501 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:15:44.638689 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:44.638696 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:44.639879 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:44.639880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:44.640825 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:44.640829 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:44.641412 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:44.641413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:44.641521 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:44.642103 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:15:44.642108 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:15:44.643590 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:15:44.645650 (+ 2060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:44.645658 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:44.645661 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:44.645889 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:44.646130 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:44.646131 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:44.646131 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:44.646246 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:44.646710 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:15:44.646717 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:15:44.646898 (+ 181us) client_negotiation.cc:769] Sending connection context
0811 05:15:44.647041 (+ 143us) client_negotiation.cc:240] Negotiation successful
0811 05:15:44.647217 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"mutex_wait_us":67,"thread_start_us":50,"threads_started":1}
I20260811 05:15:44.647393 24485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:44.637030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:38185 (local address 127.28.143.129:48125)
0811 05:15:44.637190 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:44.637203 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:44.637267 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:44.638081 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:44.638085 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:44.638503 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:15:44.638689 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:44.638696 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:44.639873 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:44.639874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:44.640825 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:44.640829 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:44.641406 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:44.641407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:44.641521 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:44.642067 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:15:44.642076 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:15:44.643942 (+ 1866us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:15:44.645919 (+ 1977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:44.645920 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:44.645920 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:44.646080 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:44.646308 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:44.646308 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:44.646309 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:44.646365 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:44.646754 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:15:44.646754 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:15:44.646898 (+ 144us) client_negotiation.cc:769] Sending connection context
0811 05:15:44.647021 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 05:15:44.647283 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":151,"thread_start_us":57,"threads_started":1}
W20260811 05:15:44.648229 23865 leader_election.cc:341] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4add4ab031844102a37662ac5da7da58 (127.28.143.131:36833): Not found: Tablet not found: 384993a5669b48769dea0a0b9824a78e
W20260811 05:15:44.648522 23857 leader_election.cc:341] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 40fd4235fcb941ba9da073d48d2599a2 (127.28.143.130:38185): Not found: Tablet not found: 384993a5669b48769dea0a0b9824a78e
I20260811 05:15:44.648583 23857 leader_election.cc:302] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1097935f8aa34158a3bda341d0425928; no voters: 40fd4235fcb941ba9da073d48d2599a2, 4add4ab031844102a37662ac5da7da58
I20260811 05:15:44.648715 24484 raft_consensus.cc:2726] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 05:15:44.922112 24485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:44.921905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.129:59339)
0811 05:15:44.922010 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:44.922068 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 05:15:44.982792 24491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:44.982499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.130:35757)
0811 05:15:44.982677 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:44.982738 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":86,"threads_started":1}
I20260811 05:15:45.116884 24492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:45.116602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.131:37119)
0811 05:15:45.116780 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:45.116832 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":118,"threads_started":1}
I20260811 05:15:45.923009 24493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:45.922693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.129:52981)
0811 05:15:45.922905 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:45.922958 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":72,"threads_started":1}
I20260811 05:15:45.983557 24494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:45.983301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.130:55313)
0811 05:15:45.983459 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:45.983509 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":68,"threads_started":1}
I20260811 05:15:46.117581 24495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:46.117363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.131:40583)
0811 05:15:46.117491 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:46.117536 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":70,"threads_started":1}
I20260811 05:15:46.184128 24496 raft_consensus.cc:482] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:15:46.184237 24496 raft_consensus.cc:504] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fd4235fcb941ba9da073d48d2599a2" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 38185 } } peers { permanent_uuid: "4add4ab031844102a37662ac5da7da58" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 36833 } } peers { permanent_uuid: "1097935f8aa34158a3bda341d0425928" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 34427 } }
I20260811 05:15:46.184427 24496 leader_election.cc:288] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 40fd4235fcb941ba9da073d48d2599a2 (127.28.143.130:38185), 4add4ab031844102a37662ac5da7da58 (127.28.143.131:36833)
W20260811 05:15:46.184926 23857 leader_election.cc:341] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 40fd4235fcb941ba9da073d48d2599a2 (127.28.143.130:38185): Not found: Tablet not found: 384993a5669b48769dea0a0b9824a78e
W20260811 05:15:46.185009 23865 leader_election.cc:341] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4add4ab031844102a37662ac5da7da58 (127.28.143.131:36833): Not found: Tablet not found: 384993a5669b48769dea0a0b9824a78e
I20260811 05:15:46.185070 23857 leader_election.cc:302] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1097935f8aa34158a3bda341d0425928; no voters: 40fd4235fcb941ba9da073d48d2599a2, 4add4ab031844102a37662ac5da7da58
I20260811 05:15:46.185161 24496 raft_consensus.cc:2726] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 05:15:46.923949 24498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:46.923582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.129:41625)
0811 05:15:46.923760 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:46.923883 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":102,"threads_started":1}
I20260811 05:15:46.984370 24500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:46.984114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.130:50361)
0811 05:15:46.984266 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:46.984320 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":70,"threads_started":1}
I20260811 05:15:47.118275 24501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:47.118043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.131:47685)
0811 05:15:47.118187 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:47.118229 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":69,"threads_started":1}
I20260811 05:15:47.740144 24502 raft_consensus.cc:482] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:15:47.740262 24502 raft_consensus.cc:504] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fd4235fcb941ba9da073d48d2599a2" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 38185 } } peers { permanent_uuid: "4add4ab031844102a37662ac5da7da58" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 36833 } } peers { permanent_uuid: "1097935f8aa34158a3bda341d0425928" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 34427 } }
I20260811 05:15:47.740448 24502 leader_election.cc:288] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 40fd4235fcb941ba9da073d48d2599a2 (127.28.143.130:38185), 4add4ab031844102a37662ac5da7da58 (127.28.143.131:36833)
W20260811 05:15:47.740857 23857 leader_election.cc:341] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 40fd4235fcb941ba9da073d48d2599a2 (127.28.143.130:38185): Not found: Tablet not found: 384993a5669b48769dea0a0b9824a78e
W20260811 05:15:47.740958 23865 leader_election.cc:341] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4add4ab031844102a37662ac5da7da58 (127.28.143.131:36833): Not found: Tablet not found: 384993a5669b48769dea0a0b9824a78e
I20260811 05:15:47.741035 23865 leader_election.cc:302] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1097935f8aa34158a3bda341d0425928; no voters: 40fd4235fcb941ba9da073d48d2599a2, 4add4ab031844102a37662ac5da7da58
I20260811 05:15:47.741127 24502 raft_consensus.cc:2726] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 05:15:47.924940 24503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:47.924591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.129:41747)
0811 05:15:47.924841 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:47.924889 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":134,"threads_started":1}
I20260811 05:15:47.985196 24505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:47.984936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.130:34327)
0811 05:15:47.985105 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:47.985147 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":85,"threads_started":1}
I20260811 05:15:48.119048 24507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:48.118795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.131:43817)
0811 05:15:48.118959 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:48.119002 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":82,"threads_started":1}
I20260811 05:15:48.925940 24509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:48.925569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.129:47381)
0811 05:15:48.925785 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:48.925837 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":97,"threads_started":1}
I20260811 05:15:48.986078 24511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:48.985819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.130:51283)
0811 05:15:48.985977 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:48.986028 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":70,"threads_started":1}
I20260811 05:15:49.119918 24515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:49.119638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.131:36897)
0811 05:15:49.119818 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:49.119867 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":81,"threads_started":1}
I20260811 05:15:49.926764 24516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:49.926514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.129:57473)
0811 05:15:49.926667 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:49.926713 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":70,"threads_started":1}
I20260811 05:15:49.986987 24517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:49.986733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.130:52003)
0811 05:15:49.986885 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:49.986926 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":80,"threads_started":1}
I20260811 05:15:50.120747 24518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:50.120460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.131:48961)
0811 05:15:50.120645 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:50.120691 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":83,"threads_started":1}
I20260811 05:15:50.474203 24528 raft_consensus.cc:482] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:15:50.474319 24528 raft_consensus.cc:504] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fd4235fcb941ba9da073d48d2599a2" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 38185 } } peers { permanent_uuid: "4add4ab031844102a37662ac5da7da58" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 36833 } } peers { permanent_uuid: "1097935f8aa34158a3bda341d0425928" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 34427 } }
I20260811 05:15:50.474534 24528 leader_election.cc:288] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 40fd4235fcb941ba9da073d48d2599a2 (127.28.143.130:38185), 4add4ab031844102a37662ac5da7da58 (127.28.143.131:36833)
W20260811 05:15:50.474924 23865 leader_election.cc:341] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4add4ab031844102a37662ac5da7da58 (127.28.143.131:36833): Not found: Tablet not found: 384993a5669b48769dea0a0b9824a78e
W20260811 05:15:50.475008 23857 leader_election.cc:341] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 40fd4235fcb941ba9da073d48d2599a2 (127.28.143.130:38185): Not found: Tablet not found: 384993a5669b48769dea0a0b9824a78e
I20260811 05:15:50.475071 23857 leader_election.cc:302] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1097935f8aa34158a3bda341d0425928; no voters: 40fd4235fcb941ba9da073d48d2599a2, 4add4ab031844102a37662ac5da7da58
I20260811 05:15:50.475142 24528 raft_consensus.cc:2726] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 05:15:50.927703 24530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:50.927436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.129:44045)
0811 05:15:50.927598 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:50.927649 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":86,"threads_started":1}
I20260811 05:15:50.987761 24532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:50.987484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.130:32793)
0811 05:15:50.987662 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:50.987711 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":81,"threads_started":1}
I20260811 05:15:51.121552 24533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:51.121307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.131:59389)
0811 05:15:51.121445 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:51.121495 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":71,"threads_started":1}
I20260811 05:15:51.928579 24536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:51.928270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.129:34791)
0811 05:15:51.928477 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:51.928528 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":117,"threads_started":1}
I20260811 05:15:51.988663 24537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:51.988327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.130:37585)
0811 05:15:51.988521 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:51.988566 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":95,"threads_started":1}
I20260811 05:15:52.122382 24538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:52.122130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.131:48937)
0811 05:15:52.122285 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:52.122333 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":67,"threads_started":1}
I20260811 05:15:52.929447 24539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:52.929167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.129:33163)
0811 05:15:52.929349 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:52.929397 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":73,"threads_started":1}
I20260811 05:15:52.989532 24540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:52.989225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.130:47313)
0811 05:15:52.989427 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:52.989476 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":101,"threads_started":1}
I20260811 05:15:53.123307 24541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:53.123061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.131:38041)
0811 05:15:53.123212 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:53.123260 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":71,"threads_started":1}
I20260811 05:15:53.930310 24543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:53.930033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.129:59745)
0811 05:15:53.930206 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:53.930258 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":90,"threads_started":1}
I20260811 05:15:53.990234 24544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:53.990029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.130:41613)
0811 05:15:53.990154 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:53.990187 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":66,"threads_started":1}
I20260811 05:15:54.124104 24545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:54.123827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.131:32773)
0811 05:15:54.124006 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:54.124054 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":121,"threads_started":1}
I20260811 05:15:54.931097 24547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:54.930851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.129:48733)
0811 05:15:54.930995 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:54.931046 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":73,"threads_started":1}
I20260811 05:15:54.991001 24548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:54.990742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.130:49609)
0811 05:15:54.990902 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:54.990949 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":72,"threads_started":1}
I20260811 05:15:55.124902 24549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:55.124675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.131:37419)
0811 05:15:55.124801 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:55.124850 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":65,"threads_started":1}
I20260811 05:15:55.931974 24550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:55.931675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.129:40345)
0811 05:15:55.931858 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:55.931915 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":79,"threads_started":1}
I20260811 05:15:55.991955 24551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:55.991662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.130:60039)
0811 05:15:55.991848 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:55.991901 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":83,"threads_started":1}
I20260811 05:15:56.125578 24552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:56.125354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.131:36771)
0811 05:15:56.125484 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:56.125521 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":63,"threads_started":1}
I20260811 05:15:56.932874 24553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:56.932555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.129:37195)
0811 05:15:56.932763 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:56.932814 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":103,"threads_started":1}
I20260811 05:15:56.992842 24554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:56.992572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.130:43403)
0811 05:15:56.992743 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:56.992789 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":117,"threads_started":1}
I20260811 05:15:57.126452 24555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:57.126161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.131:41883)
0811 05:15:57.126339 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:57.126402 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":72,"threads_started":1}
I20260811 05:15:57.653551 24557 raft_consensus.cc:482] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:15:57.653684 24557 raft_consensus.cc:504] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fd4235fcb941ba9da073d48d2599a2" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 38185 } } peers { permanent_uuid: "4add4ab031844102a37662ac5da7da58" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 36833 } } peers { permanent_uuid: "1097935f8aa34158a3bda341d0425928" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 34427 } }
I20260811 05:15:57.653874 24557 leader_election.cc:288] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 40fd4235fcb941ba9da073d48d2599a2 (127.28.143.130:38185), 4add4ab031844102a37662ac5da7da58 (127.28.143.131:36833)
W20260811 05:15:57.654299 23865 leader_election.cc:341] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4add4ab031844102a37662ac5da7da58 (127.28.143.131:36833): Not found: Tablet not found: 384993a5669b48769dea0a0b9824a78e
W20260811 05:15:57.654395 23857 leader_election.cc:341] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 40fd4235fcb941ba9da073d48d2599a2 (127.28.143.130:38185): Not found: Tablet not found: 384993a5669b48769dea0a0b9824a78e
I20260811 05:15:57.654455 23857 leader_election.cc:302] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1097935f8aa34158a3bda341d0425928; no voters: 40fd4235fcb941ba9da073d48d2599a2, 4add4ab031844102a37662ac5da7da58
I20260811 05:15:57.654538 24557 raft_consensus.cc:2726] T 384993a5669b48769dea0a0b9824a78e P 1097935f8aa34158a3bda341d0425928 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 05:15:57.933847 24559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:57.933580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.129:43483)
0811 05:15:57.933743 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:57.933798 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":70,"threads_started":1}
I20260811 05:15:57.993709 24565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:57.993457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.130:44549)
0811 05:15:57.993613 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:57.993658 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":72,"threads_started":1}
I20260811 05:15:58.118181 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 23776
I20260811 05:15:58.130896 24567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:58.130667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:43443 (local address 127.28.143.131:36885)
0811 05:15:58.130807 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:58.130850 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:43443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":69,"threads_started":1}
I20260811 05:15:58.134867 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 24038
I20260811 05:15:58.154300 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 24260
2026-08-11T05:15:58Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18655 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:15:58 dist-test-slave-f7th krb5kdc[24572](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:15:58 dist-test-slave-f7th krb5kdc[24572](info): set up 2 sockets
Aug 11 05:15:58 dist-test-slave-f7th krb5kdc[24572](info): commencing operation
krb5kdc: starting...
W20260811 05:15:59.382014 29246 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Aug 11 05:15:59 dist-test-slave-f7th krb5kdc[24572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:15:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:15:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
I20260811 05:15:59.516883 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:36473
--webserver_interface=127.28.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:42519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:36473
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:15:59.608711 24602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:15:59.608932 24602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:15:59.608965 24602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:15:59.610216 24602 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 05:15:59.610258 24602 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 05:15:59.610275 24602 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 05:15:59.610291 24602 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:15:59.610308 24602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:15:59.610334 24602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:15:59.610358 24602 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:15:59.610381 24602 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:15:59.612004 24602 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:42519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:36473
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:36473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:15:59.612277 24602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:15:59.612514 24602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:15:59.617568 24614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:59.617581 24611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:59.617715 24602 server_base.cc:1034] running on GCE node
W20260811 05:15:59.617774 24610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:59.618196 24602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:15:59.618528 24602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:15:59.619681 24602 hybrid_clock.cc:648] HybridClock initialized: now 1786425359619671 us; error 33 us; skew 500 ppm
Aug 11 05:15:59 dist-test-slave-f7th krb5kdc[24572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425359, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:15:59.622416 24602 init.cc:377] Logged in from keytab as kudu/127.28.143.190@KRBTEST.COM (short username kudu)
I20260811 05:15:59.622937 24602 webserver.cc:460] Webserver started at http://127.28.143.190:38351/ using document root <none> and password file <none>
I20260811 05:15:59.623186 24602 fs_manager.cc:359] Metadata directory not provided
I20260811 05:15:59.623237 24602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:15:59.623369 24602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:15:59.624477 24602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "763111c9cea64e29a4dc12beb1394115"
format_stamp: "Formatted at 2026-08-11 05:15:59 on dist-test-slave-f7th"
server_key: "6c5bc1fc8a07b2e380406551df49f7a5"
server_key_iv: "366ad4c1680a74560d1671bc30f58186"
server_key_version: "encryptionkey@0"
I20260811 05:15:59.624848 24602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "763111c9cea64e29a4dc12beb1394115"
format_stamp: "Formatted at 2026-08-11 05:15:59 on dist-test-slave-f7th"
server_key: "6c5bc1fc8a07b2e380406551df49f7a5"
server_key_iv: "366ad4c1680a74560d1671bc30f58186"
server_key_version: "encryptionkey@0"
I20260811 05:15:59.626222 24602 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:15:59.627019 24624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:59.627183 24602 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:15:59.627276 24602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "763111c9cea64e29a4dc12beb1394115"
format_stamp: "Formatted at 2026-08-11 05:15:59 on dist-test-slave-f7th"
server_key: "6c5bc1fc8a07b2e380406551df49f7a5"
server_key_iv: "366ad4c1680a74560d1671bc30f58186"
server_key_version: "encryptionkey@0"
I20260811 05:15:59.627341 24602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:15:59.639603 24602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:15:59.641804 24602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:15:59.641952 24602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:15:59.646142 24602 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:36473
I20260811 05:15:59.646245 24714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:36473 every 8 connection(s)
I20260811 05:15:59.646554 24602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:15:59.647493 24716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:15:59.650007 24716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115: Bootstrap starting.
I20260811 05:15:59.650651 24716 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115: Neither blocks nor log segments found. Creating new log.
I20260811 05:15:59.650939 24716 log.cc:824] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115: Log is configured to *not* fsync() on all Append() calls
I20260811 05:15:59.651685 24716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115: No bootstrap required, opened a new log
I20260811 05:15:59.653419 24716 raft_consensus.cc:348] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "763111c9cea64e29a4dc12beb1394115" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 36473 } }
I20260811 05:15:59.653577 24716 raft_consensus.cc:374] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:15:59.653599 24716 raft_consensus.cc:729] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 763111c9cea64e29a4dc12beb1394115, State: Initialized, Role: FOLLOWER
I20260811 05:15:59.653738 24716 consensus_queue.cc:260] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "763111c9cea64e29a4dc12beb1394115" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 36473 } }
I20260811 05:15:59.653808 24716 raft_consensus.cc:388] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:15:59.653834 24716 raft_consensus.cc:482] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:15:59.653883 24716 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:15:59.654426 24716 raft_consensus.cc:504] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "763111c9cea64e29a4dc12beb1394115" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 36473 } }
I20260811 05:15:59.654536 24716 leader_election.cc:302] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 763111c9cea64e29a4dc12beb1394115; no voters:
I20260811 05:15:59.654717 24716 leader_election.cc:288] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:15:59.654795 24724 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:15:59.654959 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 24602
I20260811 05:15:59.655000 24716 sys_catalog.cc:564] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:15:59.655052 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:15:59.654999 24724 raft_consensus.cc:686] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [term 1 LEADER]: Becoming Leader. State: Replica: 763111c9cea64e29a4dc12beb1394115, State: Running, Role: LEADER
I20260811 05:15:59.655160 24724 consensus_queue.cc:237] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "763111c9cea64e29a4dc12beb1394115" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 36473 } }
I20260811 05:15:59.655287 29246 external_mini_cluster.cc:1428] Setting key 4671ebd6a02d98c9aa6a4f7bf563dd8f
I20260811 05:15:59.656044 24725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "763111c9cea64e29a4dc12beb1394115" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "763111c9cea64e29a4dc12beb1394115" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 36473 } } }
I20260811 05:15:59.656147 24725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [sys.catalog]: This master's current role is: LEADER
I20260811 05:15:59.656311 24726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 763111c9cea64e29a4dc12beb1394115. Latest consensus state: current_term: 1 leader_uuid: "763111c9cea64e29a4dc12beb1394115" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "763111c9cea64e29a4dc12beb1394115" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 36473 } } }
I20260811 05:15:59.656399 24726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [sys.catalog]: This master's current role is: LEADER
I20260811 05:15:59.656530 24748 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:15:59.657634 24748 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:15:59.659478 24748 catalog_manager.cc:1347] Generated new cluster ID: 7b5ac7d58ac345428678d172a6e7d067
I20260811 05:15:59.659551 24748 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:15:59.667910 24748 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:15:59.668710 24748 catalog_manager.cc:1504] Loading token signing keys...
Aug 11 05:15:59 dist-test-slave-f7th krb5kdc[24572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:15:59.674081 24740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:59.655922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48480 (local address 127.28.143.190:36473)
0811 05:15:59.656119 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:59.656129 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:59.656646 (+ 517us) server_negotiation.cc:407] Connection header received
0811 05:15:59.656856 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:59.656862 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:59.656908 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:59.657028 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:59.657683 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:59.658865 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:59.667773 (+ 8908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:59.668359 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:59.668477 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:59.670955 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:59.670961 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:59.670961 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:59.670976 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:59.672540 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:59.672930 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:59.672934 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:59.672934 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:59.673026 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:59.673210 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:59.673211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:59.673212 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:59.673448 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:59.673621 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:59.673705 (+ 84us) server_negotiation.cc:299] Negotiation successful
0811 05:15:59.673872 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":73,"threads_started":1}
I20260811 05:15:59.677731 24748 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115: Generated new TSK 0
I20260811 05:15:59.677940 24748 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.143.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab.
Entry for principal kudu/127.28.143.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab.
I20260811 05:15:59.712459 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.129:0
--local_ip_for_outbound_sockets=127.28.143.129
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:36473
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:42519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:15:59.800701 24765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:15:59.800948 24765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:15:59.800976 24765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:15:59.802245 24765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:15:59.802301 24765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:15:59.802330 24765 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.129
I20260811 05:15:59.803920 24765 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:42519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:36473
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.129
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:15:59.804211 24765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:15:59.804463 24765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:15:59.809801 24776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:59.809783 24772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:59.809784 24774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:59.809875 24765 server_base.cc:1034] running on GCE node
I20260811 05:15:59.810245 24765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:15:59.810498 24765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:15:59.811659 24765 hybrid_clock.cc:648] HybridClock initialized: now 1786425359811646 us; error 29 us; skew 500 ppm
Aug 11 05:15:59 dist-test-slave-f7th krb5kdc[24572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425359, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:15:59.814364 24765 init.cc:377] Logged in from keytab as kudu/127.28.143.129@KRBTEST.COM (short username kudu)
I20260811 05:15:59.814862 24765 webserver.cc:460] Webserver started at http://127.28.143.129:41097/ using document root <none> and password file <none>
I20260811 05:15:59.815084 24765 fs_manager.cc:359] Metadata directory not provided
I20260811 05:15:59.815125 24765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:15:59.815220 24765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:15:59.816248 24765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/data/instance:
uuid: "95bb8596089e4e69ba1427ac129f4a0f"
format_stamp: "Formatted at 2026-08-11 05:15:59 on dist-test-slave-f7th"
server_key: "808265f319c6dda4abdb4683fab4dd79"
server_key_iv: "4e52b2f22d5584f5985c214383965cbc"
server_key_version: "encryptionkey@0"
I20260811 05:15:59.816558 24765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance:
uuid: "95bb8596089e4e69ba1427ac129f4a0f"
format_stamp: "Formatted at 2026-08-11 05:15:59 on dist-test-slave-f7th"
server_key: "808265f319c6dda4abdb4683fab4dd79"
server_key_iv: "4e52b2f22d5584f5985c214383965cbc"
server_key_version: "encryptionkey@0"
I20260811 05:15:59.817878 24765 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 05:15:59.818601 24783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:59.818760 24765 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 05:15:59.818845 24765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/wal
uuid: "95bb8596089e4e69ba1427ac129f4a0f"
format_stamp: "Formatted at 2026-08-11 05:15:59 on dist-test-slave-f7th"
server_key: "808265f319c6dda4abdb4683fab4dd79"
server_key_iv: "4e52b2f22d5584f5985c214383965cbc"
server_key_version: "encryptionkey@0"
I20260811 05:15:59.818912 24765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:15:59.827800 24765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:15:59.830157 24765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:15:59.830324 24765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:15:59.830566 24765 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:15:59.830894 24765 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:15:59.830931 24765 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:59.830960 24765 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:15:59.830978 24765 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:59.836764 24765 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.129:43851
I20260811 05:15:59.836835 24981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.129:43851 every 8 connection(s)
I20260811 05:15:59.837219 24765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
I20260811 05:15:59.838968 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 24765
I20260811 05:15:59.839079 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance
I20260811 05:15:59.839286 29246 external_mini_cluster.cc:1428] Setting key aaa84fd933ecf78e81f16ca9d09ef753
Aug 11 05:15:59 dist-test-slave-f7th krb5kdc[24572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425359, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:15:59.847822 24740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:59.838070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:48871 (local address 127.28.143.190:36473)
0811 05:15:59.838217 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:59.838228 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:59.838278 (+ 50us) server_negotiation.cc:407] Connection header received
0811 05:15:59.838863 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:59.838865 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:59.838880 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:59.838971 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:59.840482 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:59.841069 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:59.842140 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:59.842360 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:59.842426 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:59.844792 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:59.844794 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:59.844794 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:59.844803 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:59.846174 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:59.846736 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:59.846738 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:59.846738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:59.846802 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:59.847128 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:59.847128 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:59.847129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:59.847249 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:59.847299 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:59.847657 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 05:15:59.847749 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260811 05:15:59.848076 24987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:59.837973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.129:48871)
0811 05:15:59.838152 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:59.838198 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:59.838724 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:59.839032 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:59.839035 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:59.839844 (+ 809us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:15:59.840284 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:59.840293 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:59.841171 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:59.841173 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:59.842046 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:59.842049 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:59.842439 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:59.842441 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:59.842570 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:59.843097 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:15:59.843106 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:15:59.844692 (+ 1586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:15:59.846270 (+ 1578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:59.846281 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:59.846284 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:59.846628 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:59.846891 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:59.846892 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:59.846892 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:59.847054 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:59.847304 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:15:59.847312 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:15:59.847508 (+ 196us) client_negotiation.cc:769] Sending connection context
0811 05:15:59.847636 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 05:15:59.847825 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":85,"threads_started":1}
I20260811 05:15:59.848708 24983 heartbeater.cc:347] Connected to a master server at 127.28.143.190:36473
I20260811 05:15:59.848812 24983 heartbeater.cc:464] Registering TS with master...
I20260811 05:15:59.849013 24983 heartbeater.cc:511] Master 127.28.143.190:36473 requested a full tablet report, sending...
I20260811 05:15:59.849637 24648 ts_manager.cc:194] Registered new tserver with Master: 95bb8596089e4e69ba1427ac129f4a0f (127.28.143.129:43851)
WARNING: no policy specified for kudu/127.28.143.130@KRBTEST.COM; defaulting to no policy
I20260811 05:15:59.850382 24648 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.129@KRBTEST.COM'} at 127.28.143.129:48871
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab.
Entry for principal kudu/127.28.143.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab.
I20260811 05:15:59.872262 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.130:0
--local_ip_for_outbound_sockets=127.28.143.130
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:36473
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:42519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:15:59.957324 24994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:15:59.957563 24994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:15:59.957587 24994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:15:59.958789 24994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:15:59.958837 24994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:15:59.958865 24994 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.130
I20260811 05:15:59.960343 24994 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:42519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:36473
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.130
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:15:59.960566 24994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:15:59.960805 24994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:15:59.966213 25002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:59.966378 24994 server_base.cc:1034] running on GCE node
W20260811 05:15:59.966202 25005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:59.966202 25003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:59.966737 24994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:15:59.967018 24994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:15:59.968194 24994 hybrid_clock.cc:648] HybridClock initialized: now 1786425359968169 us; error 41 us; skew 500 ppm
Aug 11 05:15:59 dist-test-slave-f7th krb5kdc[24572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425359, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:15:59.970825 24994 init.cc:377] Logged in from keytab as kudu/127.28.143.130@KRBTEST.COM (short username kudu)
I20260811 05:15:59.971318 24994 webserver.cc:460] Webserver started at http://127.28.143.130:43843/ using document root <none> and password file <none>
I20260811 05:15:59.971537 24994 fs_manager.cc:359] Metadata directory not provided
I20260811 05:15:59.971582 24994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:15:59.971676 24994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:15:59.972790 24994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/data/instance:
uuid: "b9b4e2cafa624126b2b8f0f05cedde73"
format_stamp: "Formatted at 2026-08-11 05:15:59 on dist-test-slave-f7th"
server_key: "4c0c969e5ad3c05464bb43e9174b0e4a"
server_key_iv: "5ce0e781295116ec92b69284b185224f"
server_key_version: "encryptionkey@0"
I20260811 05:15:59.973124 24994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance:
uuid: "b9b4e2cafa624126b2b8f0f05cedde73"
format_stamp: "Formatted at 2026-08-11 05:15:59 on dist-test-slave-f7th"
server_key: "4c0c969e5ad3c05464bb43e9174b0e4a"
server_key_iv: "5ce0e781295116ec92b69284b185224f"
server_key_version: "encryptionkey@0"
I20260811 05:15:59.974421 24994 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 05:15:59.975184 25024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:59.975368 24994 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 05:15:59.975467 24994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/wal
uuid: "b9b4e2cafa624126b2b8f0f05cedde73"
format_stamp: "Formatted at 2026-08-11 05:15:59 on dist-test-slave-f7th"
server_key: "4c0c969e5ad3c05464bb43e9174b0e4a"
server_key_iv: "5ce0e781295116ec92b69284b185224f"
server_key_version: "encryptionkey@0"
I20260811 05:15:59.975528 24994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:15:59.991192 24994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:15:59.993453 24994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:15:59.993579 24994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:15:59.993794 24994 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:15:59.994158 24994 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:15:59.994194 24994 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:59.994217 24994 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:15:59.994240 24994 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:59.999889 24994 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.130:40211
I20260811 05:15:59.999967 25210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.130:40211 every 8 connection(s)
I20260811 05:16:00.000284 24994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
Aug 11 05:16:00 dist-test-slave-f7th krb5kdc[24572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425359, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:16:00.008766 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 24994
I20260811 05:16:00.008864 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance
I20260811 05:16:00.009207 29246 external_mini_cluster.cc:1428] Setting key 6626bcb470f9ea7e4e9169c33d612460
I20260811 05:16:00.010857 24740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:00.001384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:39021 (local address 127.28.143.190:36473)
0811 05:16:00.001466 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:00.001469 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:00.001686 (+ 217us) server_negotiation.cc:407] Connection header received
0811 05:16:00.002170 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:00.002171 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:00.002184 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:00.002252 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:00.003269 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:00.003780 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:00.004938 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:00.005136 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:00.005202 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:00.007488 (+ 2286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:00.007490 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:00.007490 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:00.007498 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:00.008957 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:00.009608 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:00.009610 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:00.009610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:00.009679 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:00.010052 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:00.010054 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:00.010054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:00.010217 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:00.010285 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:00.010695 (+ 410us) server_negotiation.cc:299] Negotiation successful
0811 05:16:00.010786 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 05:16:00.011108 25215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:00.001307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.130:39021)
0811 05:16:00.001505 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:00.001550 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:00.002067 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:00.002318 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:00.002322 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:00.002955 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:16:00.003159 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:00.003165 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:00.003893 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:00.003894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:00.004810 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:00.004813 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:00.005211 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:00.005212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:00.005326 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:00.005864 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:16:00.005873 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:16:00.007379 (+ 1506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:16:00.009048 (+ 1669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:00.009061 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:00.009064 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:00.009495 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:00.009748 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:00.009749 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:00.009750 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:00.009916 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:00.010318 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:16:00.010329 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:16:00.010522 (+ 193us) client_negotiation.cc:769] Sending connection context
0811 05:16:00.010649 (+ 127us) client_negotiation.cc:240] Negotiation successful
0811 05:16:00.010836 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":100,"threads_started":1}
I20260811 05:16:00.011613 25211 heartbeater.cc:347] Connected to a master server at 127.28.143.190:36473
I20260811 05:16:00.011725 25211 heartbeater.cc:464] Registering TS with master...
I20260811 05:16:00.011981 25211 heartbeater.cc:511] Master 127.28.143.190:36473 requested a full tablet report, sending...
I20260811 05:16:00.012482 24648 ts_manager.cc:194] Registered new tserver with Master: b9b4e2cafa624126b2b8f0f05cedde73 (127.28.143.130:40211)
I20260811 05:16:00.012923 24648 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.130@KRBTEST.COM'} at 127.28.143.130:39021
WARNING: no policy specified for kudu/127.28.143.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab.
Entry for principal kudu/127.28.143.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab.
I20260811 05:16:00.040591 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.131:0
--local_ip_for_outbound_sockets=127.28.143.131
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:36473
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:42519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:16:00.125818 25218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:16:00.126047 25218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:16:00.126082 25218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:16:00.127281 25218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:16:00.127329 25218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:16:00.127365 25218 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.131
I20260811 05:16:00.128924 25218 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:42519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:36473
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25218
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.131
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:16:00.129184 25218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:16:00.129408 25218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:16:00.134922 25226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:00.134922 25233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:00.135008 25228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:00.135221 25218 server_base.cc:1034] running on GCE node
I20260811 05:16:00.135412 25218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:16:00.135667 25218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:16:00.136849 25218 hybrid_clock.cc:648] HybridClock initialized: now 1786425360136835 us; error 32 us; skew 500 ppm
Aug 11 05:16:00 dist-test-slave-f7th krb5kdc[24572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425360, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:16:00.139671 25218 init.cc:377] Logged in from keytab as kudu/127.28.143.131@KRBTEST.COM (short username kudu)
I20260811 05:16:00.140202 25218 webserver.cc:460] Webserver started at http://127.28.143.131:37981/ using document root <none> and password file <none>
I20260811 05:16:00.140430 25218 fs_manager.cc:359] Metadata directory not provided
I20260811 05:16:00.140481 25218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:16:00.140651 25218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:16:00.141732 25218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/data/instance:
uuid: "169da5b1aeea4a0a88c8ccab57cc2f14"
format_stamp: "Formatted at 2026-08-11 05:16:00 on dist-test-slave-f7th"
server_key: "cb97f7628eb515ea0b7ef627d81e2d18"
server_key_iv: "c5767a466c0e581804665a71927a559c"
server_key_version: "encryptionkey@0"
I20260811 05:16:00.142086 25218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance:
uuid: "169da5b1aeea4a0a88c8ccab57cc2f14"
format_stamp: "Formatted at 2026-08-11 05:16:00 on dist-test-slave-f7th"
server_key: "cb97f7628eb515ea0b7ef627d81e2d18"
server_key_iv: "c5767a466c0e581804665a71927a559c"
server_key_version: "encryptionkey@0"
I20260811 05:16:00.143537 25218 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 05:16:00.144290 25248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:00.144480 25218 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:16:00.144575 25218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/wal
uuid: "169da5b1aeea4a0a88c8ccab57cc2f14"
format_stamp: "Formatted at 2026-08-11 05:16:00 on dist-test-slave-f7th"
server_key: "cb97f7628eb515ea0b7ef627d81e2d18"
server_key_iv: "c5767a466c0e581804665a71927a559c"
server_key_version: "encryptionkey@0"
I20260811 05:16:00.144678 25218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:16:00.170985 25218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:16:00.173247 25218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:16:00.173398 25218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:16:00.173660 25218 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:16:00.174005 25218 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:16:00.174041 25218 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:00.174070 25218 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:16:00.174094 25218 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:00.180204 25218 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.131:44175
I20260811 05:16:00.180534 25439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.131:44175 every 8 connection(s)
I20260811 05:16:00.180665 25218 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
Aug 11 05:16:00 dist-test-slave-f7th krb5kdc[24572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425360, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:16:00.187461 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 25218
I20260811 05:16:00.187551 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance
I20260811 05:16:00.187738 29246 external_mini_cluster.cc:1428] Setting key e1bddd48a49f3fc02154dc0df2340732
I20260811 05:16:00.190147 24740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:00.181400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:53059 (local address 127.28.143.190:36473)
0811 05:16:00.181509 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:00.181511 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:00.181666 (+ 155us) server_negotiation.cc:407] Connection header received
0811 05:16:00.182166 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:00.182168 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:00.182183 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:00.182247 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:00.183165 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:00.183733 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:00.184758 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:00.184978 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:00.185045 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:00.187386 (+ 2341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:00.187388 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:00.187388 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:00.187396 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:00.188875 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:00.189245 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:00.189246 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:00.189247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:00.189300 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:00.189509 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:00.189510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:00.189510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:00.189634 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:00.189684 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:00.190001 (+ 317us) server_negotiation.cc:299] Negotiation successful
0811 05:16:00.190079 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 05:16:00.190326 25446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:00.181352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.131:53059)
0811 05:16:00.181526 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:00.181571 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:00.182053 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:00.182302 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:00.182305 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:00.182890 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:16:00.183089 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:00.183094 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:00.183810 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:00.183811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:00.184675 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:00.184678 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:00.185089 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:00.185090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:00.185215 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:00.185754 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:16:00.185764 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:16:00.187247 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:16:00.188948 (+ 1701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:00.188955 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:00.188957 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:00.189183 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:00.189347 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:00.189347 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:00.189347 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:00.189465 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:00.189723 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:16:00.189730 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:16:00.189865 (+ 135us) client_negotiation.cc:769] Sending connection context
0811 05:16:00.189973 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 05:16:00.190112 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":80,"threads_started":1}
I20260811 05:16:00.190789 25440 heartbeater.cc:347] Connected to a master server at 127.28.143.190:36473
I20260811 05:16:00.190881 25440 heartbeater.cc:464] Registering TS with master...
I20260811 05:16:00.191072 25440 heartbeater.cc:511] Master 127.28.143.190:36473 requested a full tablet report, sending...
I20260811 05:16:00.191421 24648 ts_manager.cc:194] Registered new tserver with Master: 169da5b1aeea4a0a88c8ccab57cc2f14 (127.28.143.131:44175)
I20260811 05:16:00.191769 24648 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.131@KRBTEST.COM'} at 127.28.143.131:53059
I20260811 05:16:00.199975 29246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:16:00.207469 24740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:00.201360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48490 (local address 127.28.143.190:36473)
0811 05:16:00.201458 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:00.201460 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:00.201520 (+ 60us) server_negotiation.cc:407] Connection header received
0811 05:16:00.201566 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:00.201567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:00.201576 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:00.201629 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:00.202169 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:00.202659 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:00.203394 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:00.203589 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:00.203652 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:00.204918 (+ 1266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:00.204919 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:00.204919 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:00.204926 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:00.206153 (+ 1227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:00.206530 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:00.206532 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:00.206532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:00.206595 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:00.206870 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:00.206871 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:00.206872 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:00.207090 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:00.207182 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:00.207326 (+ 144us) server_negotiation.cc:299] Negotiation successful
0811 05:16:00.207399 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 05:16:00.209172 24648 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48490:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:16:00.210773 24648 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:16:00.217424 25466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:00.213288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.131:44175 (local address 127.0.0.1:60890)
0811 05:16:00.213505 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:00.213514 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:00.213559 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:00.213850 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:00.213854 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:00.213856 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:16:00.213974 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:00.213977 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:00.215075 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:00.215076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:00.216401 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:00.216402 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:00.217101 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:00.217102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:00.217191 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:00.217207 (+ 16us) client_negotiation.cc:769] Sending connection context
0811 05:16:00.217276 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 05:16:00.217328 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":52,"thread_start_us":74,"threads_started":1}
I20260811 05:16:00.217449 25464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:00.213214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59220 (local address 127.28.143.129:43851)
0811 05:16:00.213964 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:00.213968 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:00.213977 (+ 9us) server_negotiation.cc:407] Connection header received
0811 05:16:00.214028 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:00.214030 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:00.214045 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:00.214118 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:16:00.214351 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:00.215371 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:00.216417 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:00.217163 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:00.217238 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:00.217291 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:00.217351 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 05:16:00.217384 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":710,"thread_start_us":103,"threads_started":1}
I20260811 05:16:00.217689 25469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:00.213294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33946 (local address 127.28.143.130:40211)
0811 05:16:00.213663 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:00.213667 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:00.213678 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:16:00.213720 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:00.213722 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:00.213738 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:00.213816 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:16:00.214066 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:00.215166 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:00.216455 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:00.217163 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:00.217238 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:00.217291 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:00.217595 (+ 304us) server_negotiation.cc:299] Negotiation successful
0811 05:16:00.217631 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":78,"threads_started":1}
I20260811 05:16:00.218044 25470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:00.213485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60890 (local address 127.28.143.131:44175)
0811 05:16:00.213702 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:00.213706 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:00.213715 (+ 9us) server_negotiation.cc:407] Connection header received
0811 05:16:00.213748 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:00.213750 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:00.213765 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:00.214083 (+ 318us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:16:00.214159 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:00.214981 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:00.216490 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:00.217022 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:00.217101 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:00.217165 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:00.217940 (+ 775us) server_negotiation.cc:299] Negotiation successful
0811 05:16:00.217985 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":122,"threads_started":1}
I20260811 05:16:00.217429 25460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:00.213078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:43851 (local address 127.0.0.1:59220)
0811 05:16:00.213254 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:00.213296 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:00.213359 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:00.214207 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:00.214208 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:00.214210 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:16:00.214262 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:00.214264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:00.215456 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:00.215457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:00.216347 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:00.216349 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:00.217235 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:00.217236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:00.217290 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:00.217293 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 05:16:00.217340 (+ 47us) client_negotiation.cc:240] Negotiation successful
0811 05:16:00.217369 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":68,"threads_started":1}
I20260811 05:16:00.217659 25463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:00.213085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:40211 (local address 127.0.0.1:33946)
0811 05:16:00.213456 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:00.213488 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:00.213537 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:00.213850 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:00.213853 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:00.213858 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:16:00.213974 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:00.213977 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:00.215265 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:00.215266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:00.216382 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:00.216385 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:00.217454 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:00.217455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:00.217514 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:00.217518 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:16:00.217566 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 05:16:00.217604 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":40,"thread_start_us":73,"threads_started":1}
I20260811 05:16:00.218873 25328 tablet_service.cc:1467] Processing CreateTablet for tablet 8f45eff0b7bf4b41b603307b056177d3 (DEFAULT_TABLE table=test-table [id=d1ce7da03b3f44e4941a1d6fab7d38bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:16:00.218973 24866 tablet_service.cc:1467] Processing CreateTablet for tablet 8f45eff0b7bf4b41b603307b056177d3 (DEFAULT_TABLE table=test-table [id=d1ce7da03b3f44e4941a1d6fab7d38bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:16:00.219274 25328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f45eff0b7bf4b41b603307b056177d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:16:00.219274 24866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f45eff0b7bf4b41b603307b056177d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:16:00.221191 25095 tablet_service.cc:1467] Processing CreateTablet for tablet 8f45eff0b7bf4b41b603307b056177d3 (DEFAULT_TABLE table=test-table [id=d1ce7da03b3f44e4941a1d6fab7d38bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:16:00.221570 25095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f45eff0b7bf4b41b603307b056177d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:16:00.222880 25474 tablet_bootstrap.cc:492] T 8f45eff0b7bf4b41b603307b056177d3 P 95bb8596089e4e69ba1427ac129f4a0f: Bootstrap starting.
I20260811 05:16:00.223600 25474 tablet_bootstrap.cc:654] T 8f45eff0b7bf4b41b603307b056177d3 P 95bb8596089e4e69ba1427ac129f4a0f: Neither blocks nor log segments found. Creating new log.
I20260811 05:16:00.224202 25474 log.cc:824] T 8f45eff0b7bf4b41b603307b056177d3 P 95bb8596089e4e69ba1427ac129f4a0f: Log is configured to *not* fsync() on all Append() calls
I20260811 05:16:00.224484 25475 tablet_bootstrap.cc:492] T 8f45eff0b7bf4b41b603307b056177d3 P 169da5b1aeea4a0a88c8ccab57cc2f14: Bootstrap starting.
I20260811 05:16:00.224602 25476 tablet_bootstrap.cc:492] T 8f45eff0b7bf4b41b603307b056177d3 P b9b4e2cafa624126b2b8f0f05cedde73: Bootstrap starting.
I20260811 05:16:00.225239 25475 tablet_bootstrap.cc:654] T 8f45eff0b7bf4b41b603307b056177d3 P 169da5b1aeea4a0a88c8ccab57cc2f14: Neither blocks nor log segments found. Creating new log.
I20260811 05:16:00.225250 25476 tablet_bootstrap.cc:654] T 8f45eff0b7bf4b41b603307b056177d3 P b9b4e2cafa624126b2b8f0f05cedde73: Neither blocks nor log segments found. Creating new log.
I20260811 05:16:00.225687 25475 log.cc:824] T 8f45eff0b7bf4b41b603307b056177d3 P 169da5b1aeea4a0a88c8ccab57cc2f14: Log is configured to *not* fsync() on all Append() calls
I20260811 05:16:00.225696 25476 log.cc:824] T 8f45eff0b7bf4b41b603307b056177d3 P b9b4e2cafa624126b2b8f0f05cedde73: Log is configured to *not* fsync() on all Append() calls
I20260811 05:16:00.226409 25476 tablet_bootstrap.cc:492] T 8f45eff0b7bf4b41b603307b056177d3 P b9b4e2cafa624126b2b8f0f05cedde73: No bootstrap required, opened a new log
I20260811 05:16:00.226418 25475 tablet_bootstrap.cc:492] T 8f45eff0b7bf4b41b603307b056177d3 P 169da5b1aeea4a0a88c8ccab57cc2f14: No bootstrap required, opened a new log
I20260811 05:16:00.226490 25476 ts_tablet_manager.cc:1397] T 8f45eff0b7bf4b41b603307b056177d3 P b9b4e2cafa624126b2b8f0f05cedde73: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 05:16:00.226490 25475 ts_tablet_manager.cc:1397] T 8f45eff0b7bf4b41b603307b056177d3 P 169da5b1aeea4a0a88c8ccab57cc2f14: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260811 05:16:00.228421 25476 raft_consensus.cc:348] T 8f45eff0b7bf4b41b603307b056177d3 P b9b4e2cafa624126b2b8f0f05cedde73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9b4e2cafa624126b2b8f0f05cedde73" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 40211 } } peers { permanent_uuid: "169da5b1aeea4a0a88c8ccab57cc2f14" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44175 } } peers { permanent_uuid: "95bb8596089e4e69ba1427ac129f4a0f" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43851 } }
I20260811 05:16:00.228421 25475 raft_consensus.cc:348] T 8f45eff0b7bf4b41b603307b056177d3 P 169da5b1aeea4a0a88c8ccab57cc2f14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9b4e2cafa624126b2b8f0f05cedde73" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 40211 } } peers { permanent_uuid: "169da5b1aeea4a0a88c8ccab57cc2f14" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44175 } } peers { permanent_uuid: "95bb8596089e4e69ba1427ac129f4a0f" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43851 } }
I20260811 05:16:00.228644 25476 raft_consensus.cc:374] T 8f45eff0b7bf4b41b603307b056177d3 P b9b4e2cafa624126b2b8f0f05cedde73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:16:00.228647 25475 raft_consensus.cc:374] T 8f45eff0b7bf4b41b603307b056177d3 P 169da5b1aeea4a0a88c8ccab57cc2f14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:16:00.228698 25475 raft_consensus.cc:729] T 8f45eff0b7bf4b41b603307b056177d3 P 169da5b1aeea4a0a88c8ccab57cc2f14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 169da5b1aeea4a0a88c8ccab57cc2f14, State: Initialized, Role: FOLLOWER
I20260811 05:16:00.228726 25476 raft_consensus.cc:729] T 8f45eff0b7bf4b41b603307b056177d3 P b9b4e2cafa624126b2b8f0f05cedde73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9b4e2cafa624126b2b8f0f05cedde73, State: Initialized, Role: FOLLOWER
I20260811 05:16:00.228858 25476 consensus_queue.cc:260] T 8f45eff0b7bf4b41b603307b056177d3 P b9b4e2cafa624126b2b8f0f05cedde73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9b4e2cafa624126b2b8f0f05cedde73" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 40211 } } peers { permanent_uuid: "169da5b1aeea4a0a88c8ccab57cc2f14" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44175 } } peers { permanent_uuid: "95bb8596089e4e69ba1427ac129f4a0f" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43851 } }
I20260811 05:16:00.228897 25475 consensus_queue.cc:260] T 8f45eff0b7bf4b41b603307b056177d3 P 169da5b1aeea4a0a88c8ccab57cc2f14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9b4e2cafa624126b2b8f0f05cedde73" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 40211 } } peers { permanent_uuid: "169da5b1aeea4a0a88c8ccab57cc2f14" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44175 } } peers { permanent_uuid: "95bb8596089e4e69ba1427ac129f4a0f" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43851 } }
I20260811 05:16:00.229040 25474 tablet_bootstrap.cc:492] T 8f45eff0b7bf4b41b603307b056177d3 P 95bb8596089e4e69ba1427ac129f4a0f: No bootstrap required, opened a new log
I20260811 05:16:00.229175 25474 ts_tablet_manager.cc:1397] T 8f45eff0b7bf4b41b603307b056177d3 P 95bb8596089e4e69ba1427ac129f4a0f: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260811 05:16:00.229203 25475 ts_tablet_manager.cc:1428] T 8f45eff0b7bf4b41b603307b056177d3 P 169da5b1aeea4a0a88c8ccab57cc2f14: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 05:16:00.229259 25476 ts_tablet_manager.cc:1428] T 8f45eff0b7bf4b41b603307b056177d3 P b9b4e2cafa624126b2b8f0f05cedde73: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260811 05:16:00.229409 25440 heartbeater.cc:503] Master 127.28.143.190:36473 was elected leader, sending a full tablet report...
I20260811 05:16:00.229719 25211 heartbeater.cc:503] Master 127.28.143.190:36473 was elected leader, sending a full tablet report...
I20260811 05:16:00.231083 25474 raft_consensus.cc:348] T 8f45eff0b7bf4b41b603307b056177d3 P 95bb8596089e4e69ba1427ac129f4a0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9b4e2cafa624126b2b8f0f05cedde73" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 40211 } } peers { permanent_uuid: "169da5b1aeea4a0a88c8ccab57cc2f14" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44175 } } peers { permanent_uuid: "95bb8596089e4e69ba1427ac129f4a0f" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43851 } }
I20260811 05:16:00.231225 25474 raft_consensus.cc:374] T 8f45eff0b7bf4b41b603307b056177d3 P 95bb8596089e4e69ba1427ac129f4a0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:16:00.231247 25474 raft_consensus.cc:729] T 8f45eff0b7bf4b41b603307b056177d3 P 95bb8596089e4e69ba1427ac129f4a0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95bb8596089e4e69ba1427ac129f4a0f, State: Initialized, Role: FOLLOWER
I20260811 05:16:00.231364 25474 consensus_queue.cc:260] T 8f45eff0b7bf4b41b603307b056177d3 P 95bb8596089e4e69ba1427ac129f4a0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9b4e2cafa624126b2b8f0f05cedde73" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 40211 } } peers { permanent_uuid: "169da5b1aeea4a0a88c8ccab57cc2f14" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44175 } } peers { permanent_uuid: "95bb8596089e4e69ba1427ac129f4a0f" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43851 } }
I20260811 05:16:00.231648 25474 ts_tablet_manager.cc:1428] T 8f45eff0b7bf4b41b603307b056177d3 P 95bb8596089e4e69ba1427ac129f4a0f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 05:16:00.231683 24983 heartbeater.cc:503] Master 127.28.143.190:36473 was elected leader, sending a full tablet report...
W20260811 05:16:00.251102 25214 tablet.cc:2369] T 8f45eff0b7bf4b41b603307b056177d3 P b9b4e2cafa624126b2b8f0f05cedde73: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 05:16:00.338011 24986 tablet.cc:2369] T 8f45eff0b7bf4b41b603307b056177d3 P 95bb8596089e4e69ba1427ac129f4a0f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 05:16:00.431295 25444 tablet.cc:2369] T 8f45eff0b7bf4b41b603307b056177d3 P 169da5b1aeea4a0a88c8ccab57cc2f14: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:16:00.467804 25481 raft_consensus.cc:482] T 8f45eff0b7bf4b41b603307b056177d3 P b9b4e2cafa624126b2b8f0f05cedde73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:16:00.467981 25481 raft_consensus.cc:504] T 8f45eff0b7bf4b41b603307b056177d3 P b9b4e2cafa624126b2b8f0f05cedde73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9b4e2cafa624126b2b8f0f05cedde73" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 40211 } } peers { permanent_uuid: "169da5b1aeea4a0a88c8ccab57cc2f14" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44175 } } peers { permanent_uuid: "95bb8596089e4e69ba1427ac129f4a0f" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43851 } }
I20260811 05:16:00.468398 25481 leader_election.cc:288] T 8f45eff0b7bf4b41b603307b056177d3 P b9b4e2cafa624126b2b8f0f05cedde73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 169da5b1aeea4a0a88c8ccab57cc2f14 (127.28.143.131:44175), 95bb8596089e4e69ba1427ac129f4a0f (127.28.143.129:43851)
I20260811 05:16:00.471329 25470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:00.468741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:50299 (local address 127.28.143.131:44175)
0811 05:16:00.468804 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:00.468806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:00.468816 (+ 10us) server_negotiation.cc:407] Connection header received
0811 05:16:00.468845 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:00.468846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:00.468857 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:00.468915 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:16:00.469150 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:00.469642 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:00.470592 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:00.470984 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:00.471048 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:00.471058 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:00.471223 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 05:16:00.471260 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 05:16:00.471329 25215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:00.468645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.131:44175 (local address 127.28.143.130:50299)
0811 05:16:00.468696 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:00.468709 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:00.468756 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:00.468960 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:00.468961 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:00.468964 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:16:00.469035 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:00.469036 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:00.469721 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:00.469722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:00.470488 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:00.470491 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:00.471090 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:00.471091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:00.471156 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:00.471159 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 05:16:00.471205 (+ 46us) client_negotiation.cc:240] Negotiation successful
0811 05:16:00.471249 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260811 05:16:00.471419 25464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:00.468716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:35651 (local address 127.28.143.129:43851)
0811 05:16:00.468773 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:00.468775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:00.469092 (+ 317us) server_negotiation.cc:407] Connection header received
0811 05:16:00.469125 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:00.469126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:00.469136 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:00.469182 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:16:00.469318 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:00.469773 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:00.470728 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:00.471101 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:00.471164 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:00.471173 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:00.471309 (+ 136us) server_negotiation.cc:299] Negotiation successful
0811 05:16:00.471348 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 05:16:00.471468 25493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:00.468647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:43851 (local address 127.28.143.130:35651)
0811 05:16:00.469026 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:00.469034 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:00.469082 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:00.469195 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:00.469196 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:00.469197 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:16:00.469264 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:00.469265 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:00.469886 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:00.469887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:00.470637 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:00.470639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:00.471183 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:00.471184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:00.471247 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:00.471251 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:16:00.471296 (+ 45us) client_negotiation.cc:240] Negotiation successful
0811 05:16:00.471335 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"mutex_wait_us":172,"thread_start_us":151,"threads_started":1}
I20260811 05:16:00.471777 24899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f45eff0b7bf4b41b603307b056177d3" candidate_uuid: "b9b4e2cafa624126b2b8f0f05cedde73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95bb8596089e4e69ba1427ac129f4a0f" is_pre_election: true
I20260811 05:16:00.471774 25359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f45eff0b7bf4b41b603307b056177d3" candidate_uuid: "b9b4e2cafa624126b2b8f0f05cedde73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "169da5b1aeea4a0a88c8ccab57cc2f14" is_pre_election: true
I20260811 05:16:00.471956 25359 raft_consensus.cc:2454] T 8f45eff0b7bf4b41b603307b056177d3 P 169da5b1aeea4a0a88c8ccab57cc2f14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9b4e2cafa624126b2b8f0f05cedde73 in term 0.
I20260811 05:16:00.471956 24899 raft_consensus.cc:2454] T 8f45eff0b7bf4b41b603307b056177d3 P 95bb8596089e4e69ba1427ac129f4a0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9b4e2cafa624126b2b8f0f05cedde73 in term 0.
I20260811 05:16:00.472312 25027 leader_election.cc:302] T 8f45eff0b7bf4b41b603307b056177d3 P b9b4e2cafa624126b2b8f0f05cedde73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 169da5b1aeea4a0a88c8ccab57cc2f14, b9b4e2cafa624126b2b8f0f05cedde73; no voters:
I20260811 05:16:00.472472 25481 raft_consensus.cc:2781] T 8f45eff0b7bf4b41b603307b056177d3 P b9b4e2cafa624126b2b8f0f05cedde73 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:16:00.472564 25481 raft_consensus.cc:482] T 8f45eff0b7bf4b41b603307b056177d3 P b9b4e2cafa624126b2b8f0f05cedde73 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:16:00.472590 25481 raft_consensus.cc:3037] T 8f45eff0b7bf4b41b603307b056177d3 P b9b4e2cafa624126b2b8f0f05cedde73 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:16:00.473469 25481 raft_consensus.cc:504] T 8f45eff0b7bf4b41b603307b056177d3 P b9b4e2cafa624126b2b8f0f05cedde73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9b4e2cafa624126b2b8f0f05cedde73" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 40211 } } peers { permanent_uuid: "169da5b1aeea4a0a88c8ccab57cc2f14" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44175 } } peers { permanent_uuid: "95bb8596089e4e69ba1427ac129f4a0f" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43851 } }
I20260811 05:16:00.473625 25481 leader_election.cc:288] T 8f45eff0b7bf4b41b603307b056177d3 P b9b4e2cafa624126b2b8f0f05cedde73 [CANDIDATE]: Term 1 election: Requested vote from peers 169da5b1aeea4a0a88c8ccab57cc2f14 (127.28.143.131:44175), 95bb8596089e4e69ba1427ac129f4a0f (127.28.143.129:43851)
I20260811 05:16:00.473878 25359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f45eff0b7bf4b41b603307b056177d3" candidate_uuid: "b9b4e2cafa624126b2b8f0f05cedde73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "169da5b1aeea4a0a88c8ccab57cc2f14"
I20260811 05:16:00.473910 24899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f45eff0b7bf4b41b603307b056177d3" candidate_uuid: "b9b4e2cafa624126b2b8f0f05cedde73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95bb8596089e4e69ba1427ac129f4a0f"
I20260811 05:16:00.473989 25359 raft_consensus.cc:3037] T 8f45eff0b7bf4b41b603307b056177d3 P 169da5b1aeea4a0a88c8ccab57cc2f14 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:16:00.473989 24899 raft_consensus.cc:3037] T 8f45eff0b7bf4b41b603307b056177d3 P 95bb8596089e4e69ba1427ac129f4a0f [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:16:00.474922 25359 raft_consensus.cc:2454] T 8f45eff0b7bf4b41b603307b056177d3 P 169da5b1aeea4a0a88c8ccab57cc2f14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9b4e2cafa624126b2b8f0f05cedde73 in term 1.
I20260811 05:16:00.474941 24899 raft_consensus.cc:2454] T 8f45eff0b7bf4b41b603307b056177d3 P 95bb8596089e4e69ba1427ac129f4a0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9b4e2cafa624126b2b8f0f05cedde73 in term 1.
I20260811 05:16:00.475152 25025 leader_election.cc:302] T 8f45eff0b7bf4b41b603307b056177d3 P b9b4e2cafa624126b2b8f0f05cedde73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95bb8596089e4e69ba1427ac129f4a0f, b9b4e2cafa624126b2b8f0f05cedde73; no voters:
I20260811 05:16:00.475292 25481 raft_consensus.cc:2781] T 8f45eff0b7bf4b41b603307b056177d3 P b9b4e2cafa624126b2b8f0f05cedde73 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:16:00.475451 25481 raft_consensus.cc:686] T 8f45eff0b7bf4b41b603307b056177d3 P b9b4e2cafa624126b2b8f0f05cedde73 [term 1 LEADER]: Becoming Leader. State: Replica: b9b4e2cafa624126b2b8f0f05cedde73, State: Running, Role: LEADER
I20260811 05:16:00.475682 25481 consensus_queue.cc:237] T 8f45eff0b7bf4b41b603307b056177d3 P b9b4e2cafa624126b2b8f0f05cedde73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9b4e2cafa624126b2b8f0f05cedde73" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 40211 } } peers { permanent_uuid: "169da5b1aeea4a0a88c8ccab57cc2f14" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44175 } } peers { permanent_uuid: "95bb8596089e4e69ba1427ac129f4a0f" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43851 } }
I20260811 05:16:00.476814 24648 catalog_manager.cc:5495] T 8f45eff0b7bf4b41b603307b056177d3 P b9b4e2cafa624126b2b8f0f05cedde73 reported cstate change: term changed from 0 to 1, leader changed from <none> to b9b4e2cafa624126b2b8f0f05cedde73 (127.28.143.130). New cstate: current_term: 1 leader_uuid: "b9b4e2cafa624126b2b8f0f05cedde73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9b4e2cafa624126b2b8f0f05cedde73" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 40211 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "169da5b1aeea4a0a88c8ccab57cc2f14" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95bb8596089e4e69ba1427ac129f4a0f" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43851 } health_report { overall_health: UNKNOWN } } }
I20260811 05:16:00.502602 25469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:00.500185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33948 (local address 127.28.143.130:40211)
0811 05:16:00.500272 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:00.500274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:00.500282 (+ 8us) server_negotiation.cc:407] Connection header received
0811 05:16:00.500320 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:00.500321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:00.500342 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:00.500407 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:16:00.500520 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:00.501069 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:00.501881 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:00.502087 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:00.502151 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:00.502289 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:16:00.502399 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:16:00.502460 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:00.502475 (+ 15us) server_negotiation.cc:299] Negotiation successful
0811 05:16:00.502495 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 05:16:00.504536 24899 raft_consensus.cc:1261] T 8f45eff0b7bf4b41b603307b056177d3 P 95bb8596089e4e69ba1427ac129f4a0f [term 1 FOLLOWER]: Refusing update from remote peer b9b4e2cafa624126b2b8f0f05cedde73: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:16:00.504556 25359 raft_consensus.cc:1261] T 8f45eff0b7bf4b41b603307b056177d3 P 169da5b1aeea4a0a88c8ccab57cc2f14 [term 1 FOLLOWER]: Refusing update from remote peer b9b4e2cafa624126b2b8f0f05cedde73: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:16:00.505067 25481 consensus_queue.cc:1035] T 8f45eff0b7bf4b41b603307b056177d3 P b9b4e2cafa624126b2b8f0f05cedde73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95bb8596089e4e69ba1427ac129f4a0f" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:16:00.505270 25494 consensus_queue.cc:1035] T 8f45eff0b7bf4b41b603307b056177d3 P b9b4e2cafa624126b2b8f0f05cedde73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "169da5b1aeea4a0a88c8ccab57cc2f14" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:16:00.507704 25497 mvcc.cc:205] Tried to move back new op lower bound from 7317198276623601664 to 7317198276509298688. Current Snapshot: MvccSnapshot[applied={T|T < 7317198276623601664}]
I20260811 05:16:00.507839 25499 mvcc.cc:205] Tried to move back new op lower bound from 7317198276623601664 to 7317198276509298688. Current Snapshot: MvccSnapshot[applied={T|T < 7317198276623601664}]
I20260811 05:16:00.510226 24648 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48490:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:16:00.510331 24648 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48490:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:16:00.511317 24648 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115: Sending DeleteTablet for 3 replicas of tablet 8f45eff0b7bf4b41b603307b056177d3
I20260811 05:16:00.511675 25095 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f45eff0b7bf4b41b603307b056177d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:16:00 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:33946
I20260811 05:16:00.511760 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 24765
I20260811 05:16:00.511750 24866 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f45eff0b7bf4b41b603307b056177d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:16:00 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:59220
I20260811 05:16:00.521483 25328 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f45eff0b7bf4b41b603307b056177d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:16:00 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:60890
I20260811 05:16:00.527669 25514 tablet_replica.cc:330] T 8f45eff0b7bf4b41b603307b056177d3 P 169da5b1aeea4a0a88c8ccab57cc2f14: stopping tablet replica
I20260811 05:16:00.527899 25514 raft_consensus.cc:2229] T 8f45eff0b7bf4b41b603307b056177d3 P 169da5b1aeea4a0a88c8ccab57cc2f14 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:16:00.528096 25514 raft_consensus.cc:2258] T 8f45eff0b7bf4b41b603307b056177d3 P 169da5b1aeea4a0a88c8ccab57cc2f14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:16:00.528312 25511 tablet_replica.cc:330] T 8f45eff0b7bf4b41b603307b056177d3 P b9b4e2cafa624126b2b8f0f05cedde73: stopping tablet replica
I20260811 05:16:00.528460 25511 raft_consensus.cc:2229] T 8f45eff0b7bf4b41b603307b056177d3 P b9b4e2cafa624126b2b8f0f05cedde73 [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:16:00.528595 25511 raft_consensus.cc:2258] T 8f45eff0b7bf4b41b603307b056177d3 P b9b4e2cafa624126b2b8f0f05cedde73 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:16:00.534318 25511 ts_tablet_manager.cc:1905] T 8f45eff0b7bf4b41b603307b056177d3 P b9b4e2cafa624126b2b8f0f05cedde73: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:16:00.536803 25511 ts_tablet_manager.cc:1918] T 8f45eff0b7bf4b41b603307b056177d3 P b9b4e2cafa624126b2b8f0f05cedde73: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:16:00.537035 25511 log.cc:1196] T 8f45eff0b7bf4b41b603307b056177d3 P b9b4e2cafa624126b2b8f0f05cedde73: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/wal/wals/8f45eff0b7bf4b41b603307b056177d3
I20260811 05:16:00.537479 25511 ts_tablet_manager.cc:1939] T 8f45eff0b7bf4b41b603307b056177d3 P b9b4e2cafa624126b2b8f0f05cedde73: Deleting consensus metadata
I20260811 05:16:00.538218 24627 catalog_manager.cc:4841] TS b9b4e2cafa624126b2b8f0f05cedde73 (127.28.143.130:40211): tablet 8f45eff0b7bf4b41b603307b056177d3 (table test-table [id=d1ce7da03b3f44e4941a1d6fab7d38bf]) successfully deleted
I20260811 05:16:00.542768 25514 ts_tablet_manager.cc:1905] T 8f45eff0b7bf4b41b603307b056177d3 P 169da5b1aeea4a0a88c8ccab57cc2f14: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:16:00.547314 25514 ts_tablet_manager.cc:1918] T 8f45eff0b7bf4b41b603307b056177d3 P 169da5b1aeea4a0a88c8ccab57cc2f14: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:16:00.547484 25514 log.cc:1196] T 8f45eff0b7bf4b41b603307b056177d3 P 169da5b1aeea4a0a88c8ccab57cc2f14: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/wal/wals/8f45eff0b7bf4b41b603307b056177d3
I20260811 05:16:00.547819 25514 ts_tablet_manager.cc:1939] T 8f45eff0b7bf4b41b603307b056177d3 P 169da5b1aeea4a0a88c8ccab57cc2f14: Deleting consensus metadata
I20260811 05:16:00.548417 24627 catalog_manager.cc:4841] TS 169da5b1aeea4a0a88c8ccab57cc2f14 (127.28.143.131:44175): tablet 8f45eff0b7bf4b41b603307b056177d3 (table test-table [id=d1ce7da03b3f44e4941a1d6fab7d38bf]) successfully deleted
W20260811 05:16:00.551786 24625 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.143.129:43851 (error 108)
I20260811 05:16:00.552379 25463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:00.552244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:43851 (local address 127.0.0.1:59236)
0811 05:16:00.552302 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:00.552345 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.129:43851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 05:16:00.552457 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 24994
W20260811 05:16:00.562645 24625 catalog_manager.cc:4584] TS 95bb8596089e4e69ba1427ac129f4a0f (127.28.143.129:43851): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8f45eff0b7bf4b41b603307b056177d3: Network error: Client connection negotiation failed: client connection to 127.28.143.129:43851: connect: Connection refused (error 111)
I20260811 05:16:00.569365 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 25218
I20260811 05:16:00.582931 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 24602
I20260811 05:16:20.598040 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:36473
--webserver_interface=127.28.143.190
--webserver_port=38351
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:42519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:36473
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:16:20.688064 25516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:16:20.688319 25516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:16:20.688345 25516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:16:20.689751 25516 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 05:16:20.689812 25516 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 05:16:20.689826 25516 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 05:16:20.689836 25516 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:16:20.689848 25516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:16:20.689862 25516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:16:20.689880 25516 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:16:20.689893 25516 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:16:20.691679 25516 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:42519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:36473
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:36473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=38351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:16:20.691995 25516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:16:20.692269 25516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:16:20.703308 25531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:20.703311 25533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:20.703341 25516 server_base.cc:1034] running on GCE node
W20260811 05:16:20.703312 25536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:20.703838 25516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:16:20.704221 25516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:16:20.705430 25516 hybrid_clock.cc:648] HybridClock initialized: now 1786425380705399 us; error 49 us; skew 500 ppm
Aug 11 05:16:20 dist-test-slave-f7th krb5kdc[24572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425380, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:16:20.708578 25516 init.cc:377] Logged in from keytab as kudu/127.28.143.190@KRBTEST.COM (short username kudu)
I20260811 05:16:20.709244 25516 webserver.cc:460] Webserver started at http://127.28.143.190:38351/ using document root <none> and password file <none>
I20260811 05:16:20.709486 25516 fs_manager.cc:359] Metadata directory not provided
I20260811 05:16:20.709559 25516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:16:20.711133 25516 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 05:16:20.711942 25550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:20.712178 25516 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:16:20.712313 25516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "763111c9cea64e29a4dc12beb1394115"
format_stamp: "Formatted at 2026-08-11 05:15:59 on dist-test-slave-f7th"
server_key: "6c5bc1fc8a07b2e380406551df49f7a5"
server_key_iv: "366ad4c1680a74560d1671bc30f58186"
server_key_version: "encryptionkey@0"
I20260811 05:16:20.712823 25516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:16:20.726045 25516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:16:20.728832 25516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:16:20.729000 25516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:16:20.733878 25516 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:36473
I20260811 05:16:20.733927 25648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:36473 every 8 connection(s)
I20260811 05:16:20.734534 25516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:16:20.736248 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 25516
I20260811 05:16:20.736271 25651 sys_catalog.cc:263] Verifying existing consensus state
I20260811 05:16:20.736589 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.129:43851
--local_ip_for_outbound_sockets=127.28.143.129
--tserver_master_addrs=127.28.143.190:36473
--webserver_port=41097
--webserver_interface=127.28.143.129
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:42519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:16:20.737358 25651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115: Bootstrap starting.
I20260811 05:16:20.747036 25651 log.cc:824] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115: Log is configured to *not* fsync() on all Append() calls
I20260811 05:16:20.749167 25651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 05:16:20.749503 25651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115: Bootstrap complete.
I20260811 05:16:20.751793 25651 raft_consensus.cc:348] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "763111c9cea64e29a4dc12beb1394115" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 36473 } }
I20260811 05:16:20.752054 25651 raft_consensus.cc:729] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 763111c9cea64e29a4dc12beb1394115, State: Initialized, Role: FOLLOWER
I20260811 05:16:20.752223 25651 consensus_queue.cc:260] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "763111c9cea64e29a4dc12beb1394115" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 36473 } }
I20260811 05:16:20.752305 25651 raft_consensus.cc:388] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:16:20.752372 25651 raft_consensus.cc:482] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:16:20.752424 25651 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [term 1 FOLLOWER]: Advancing to term 2
I20260811 05:16:20.753612 25651 raft_consensus.cc:504] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "763111c9cea64e29a4dc12beb1394115" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 36473 } }
I20260811 05:16:20.753732 25651 leader_election.cc:302] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 763111c9cea64e29a4dc12beb1394115; no voters:
I20260811 05:16:20.753917 25651 leader_election.cc:288] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 05:16:20.754019 25661 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 05:16:20.754244 25651 sys_catalog.cc:564] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:16:20.754287 25661 raft_consensus.cc:686] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [term 2 LEADER]: Becoming Leader. State: Replica: 763111c9cea64e29a4dc12beb1394115, State: Running, Role: LEADER
I20260811 05:16:20.754479 25661 consensus_queue.cc:237] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "763111c9cea64e29a4dc12beb1394115" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 36473 } }
I20260811 05:16:20.754809 25662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "763111c9cea64e29a4dc12beb1394115" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "763111c9cea64e29a4dc12beb1394115" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 36473 } } }
I20260811 05:16:20.754889 25662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [sys.catalog]: This master's current role is: LEADER
I20260811 05:16:20.755405 25663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 763111c9cea64e29a4dc12beb1394115. Latest consensus state: current_term: 2 leader_uuid: "763111c9cea64e29a4dc12beb1394115" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "763111c9cea64e29a4dc12beb1394115" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 36473 } } }
I20260811 05:16:20.755473 25663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [sys.catalog]: This master's current role is: LEADER
I20260811 05:16:20.756172 25685 catalog_manager.cc:1259] Loaded cluster ID: 7b5ac7d58ac345428678d172a6e7d067
I20260811 05:16:20.756217 25685 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115: loading cluster ID for follower catalog manager: success
I20260811 05:16:20.757063 25685 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115: acquiring CA information for follower catalog manager: success
I20260811 05:16:20.757414 25685 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 05:16:20.757483 25674 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:16:20.757929 25674 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:16:20.758015 25674 catalog_manager.cc:1259] Loaded cluster ID: 7b5ac7d58ac345428678d172a6e7d067
I20260811 05:16:20.758036 25674 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:16:20.758199 25674 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:16:20.758278 25674 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115: Loaded TSK: 0
I20260811 05:16:20.768246 25674 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115: Generated new TSK 1
I20260811 05:16:20.768508 25674 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 05:16:20.830909 25653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:16:20.831212 25653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:16:20.831239 25653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:16:20.832546 25653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:16:20.832598 25653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:16:20.832671 25653 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.129
I20260811 05:16:20.834445 25653 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:42519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.129:43851
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.143.129
--webserver_port=41097
--tserver_master_addrs=127.28.143.190:36473
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25653
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.129
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:16:20.834728 25653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:16:20.834997 25653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:16:20.841440 25696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:20.841418 25694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:20.841625 25653 server_base.cc:1034] running on GCE node
W20260811 05:16:20.841418 25702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:20.841976 25653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:16:20.842254 25653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:16:20.843427 25653 hybrid_clock.cc:648] HybridClock initialized: now 1786425380843385 us; error 35 us; skew 500 ppm
Aug 11 05:16:20 dist-test-slave-f7th krb5kdc[24572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425380, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:16:20.846374 25653 init.cc:377] Logged in from keytab as kudu/127.28.143.129@KRBTEST.COM (short username kudu)
I20260811 05:16:20.846894 25653 webserver.cc:460] Webserver started at http://127.28.143.129:41097/ using document root <none> and password file <none>
I20260811 05:16:20.847165 25653 fs_manager.cc:359] Metadata directory not provided
I20260811 05:16:20.847236 25653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:16:20.848713 25653 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 05:16:20.849500 25717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:20.849713 25653 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:16:20.849812 25653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/wal
uuid: "95bb8596089e4e69ba1427ac129f4a0f"
format_stamp: "Formatted at 2026-08-11 05:15:59 on dist-test-slave-f7th"
server_key: "808265f319c6dda4abdb4683fab4dd79"
server_key_iv: "4e52b2f22d5584f5985c214383965cbc"
server_key_version: "encryptionkey@0"
I20260811 05:16:20.850210 25653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:16:20.864231 25653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:16:20.866509 25653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:16:20.866665 25653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:16:20.866910 25653 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:16:20.867422 25731 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 05:16:20.869020 25653 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 05:16:20.869082 25653 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260811 05:16:20.869110 25653 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 05:16:20.869804 25653 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 05:16:20.869853 25653 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260811 05:16:20.869925 25731 tablet_bootstrap.cc:492] T 8f45eff0b7bf4b41b603307b056177d3 P 95bb8596089e4e69ba1427ac129f4a0f: Bootstrap starting.
I20260811 05:16:20.877728 25653 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.129:43851
I20260811 05:16:20.877879 25919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.129:43851 every 8 connection(s)
I20260811 05:16:20.878479 25653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
I20260811 05:16:20.881836 25731 log.cc:824] T 8f45eff0b7bf4b41b603307b056177d3 P 95bb8596089e4e69ba1427ac129f4a0f: Log is configured to *not* fsync() on all Append() calls
I20260811 05:16:20.883929 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 25653
I20260811 05:16:20.884212 25731 tablet_bootstrap.cc:492] T 8f45eff0b7bf4b41b603307b056177d3 P 95bb8596089e4e69ba1427ac129f4a0f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 05:16:20.884253 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.130:40211
--local_ip_for_outbound_sockets=127.28.143.130
--tserver_master_addrs=127.28.143.190:36473
--webserver_port=43843
--webserver_interface=127.28.143.130
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:42519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:16:20.884560 25731 tablet_bootstrap.cc:492] T 8f45eff0b7bf4b41b603307b056177d3 P 95bb8596089e4e69ba1427ac129f4a0f: Bootstrap complete.
I20260811 05:16:20.885674 25731 ts_tablet_manager.cc:1397] T 8f45eff0b7bf4b41b603307b056177d3 P 95bb8596089e4e69ba1427ac129f4a0f: Time spent bootstrapping tablet: real 0.016s user 0.014s sys 0.000s
I20260811 05:16:20.887280 25731 raft_consensus.cc:348] T 8f45eff0b7bf4b41b603307b056177d3 P 95bb8596089e4e69ba1427ac129f4a0f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9b4e2cafa624126b2b8f0f05cedde73" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 40211 } } peers { permanent_uuid: "169da5b1aeea4a0a88c8ccab57cc2f14" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44175 } } peers { permanent_uuid: "95bb8596089e4e69ba1427ac129f4a0f" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43851 } }
I20260811 05:16:20.887641 25731 raft_consensus.cc:729] T 8f45eff0b7bf4b41b603307b056177d3 P 95bb8596089e4e69ba1427ac129f4a0f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95bb8596089e4e69ba1427ac129f4a0f, State: Initialized, Role: FOLLOWER
I20260811 05:16:20.887776 25731 consensus_queue.cc:260] T 8f45eff0b7bf4b41b603307b056177d3 P 95bb8596089e4e69ba1427ac129f4a0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9b4e2cafa624126b2b8f0f05cedde73" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 40211 } } peers { permanent_uuid: "169da5b1aeea4a0a88c8ccab57cc2f14" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44175 } } peers { permanent_uuid: "95bb8596089e4e69ba1427ac129f4a0f" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43851 } }
I20260811 05:16:20.888068 25731 ts_tablet_manager.cc:1428] T 8f45eff0b7bf4b41b603307b056177d3 P 95bb8596089e4e69ba1427ac129f4a0f: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
Aug 11 05:16:20 dist-test-slave-f7th krb5kdc[24572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425380, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:16:20.893525 25924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:20.879295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.129:42707)
0811 05:16:20.879489 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:20.879544 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:20.880179 (+ 635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:20.881376 (+ 1197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:20.881381 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:20.881755 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:16:20.882039 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:20.882047 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:20.883708 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:20.883710 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:20.885049 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:20.885054 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:20.886116 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:20.886118 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:20.886275 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:20.886914 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:16:20.886925 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:16:20.888842 (+ 1917us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:16:20.891223 (+ 2381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:20.891236 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:20.891240 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:20.891607 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:20.892021 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:20.892022 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:20.892022 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:20.892186 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:20.892786 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:16:20.892796 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:16:20.892983 (+ 187us) client_negotiation.cc:769] Sending connection context
0811 05:16:20.893106 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 05:16:20.893298 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":99,"threads_started":1}
I20260811 05:16:20.893565 25926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:20.879450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:42707 (local address 127.28.143.190:36473)
0811 05:16:20.880787 (+ 1337us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:20.880799 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:20.880851 (+ 52us) server_negotiation.cc:407] Connection header received
0811 05:16:20.881047 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:20.881056 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:20.881133 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:20.881315 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:20.882193 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:20.883556 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:20.885174 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:20.885852 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:20.885969 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:20.889001 (+ 3032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:20.889006 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:20.889007 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:20.889021 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:20.891084 (+ 2063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:20.891717 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:20.891722 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:20.891722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:20.891875 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:20.892275 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:20.892276 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:20.892277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:20.892674 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:20.892778 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:20.893193 (+ 415us) server_negotiation.cc:299] Negotiation successful
0811 05:16:20.893357 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1260,"thread_start_us":85,"threads_started":1}
I20260811 05:16:20.894891 25921 heartbeater.cc:347] Connected to a master server at 127.28.143.190:36473
I20260811 05:16:20.895011 25921 heartbeater.cc:464] Registering TS with master...
I20260811 05:16:20.895224 25921 heartbeater.cc:511] Master 127.28.143.190:36473 requested a full tablet report, sending...
I20260811 05:16:20.896297 25578 ts_manager.cc:194] Registered new tserver with Master: 95bb8596089e4e69ba1427ac129f4a0f (127.28.143.129:43851)
I20260811 05:16:20.897735 25578 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.129@KRBTEST.COM'} at 127.28.143.129:42707
I20260811 05:16:20.898931 25921 heartbeater.cc:503] Master 127.28.143.190:36473 was elected leader, sending a full tablet report...
I20260811 05:16:20.904520 25933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:20.897149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:43851 (local address 127.0.0.1:45016)
0811 05:16:20.900741 (+ 3592us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:20.900795 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:20.900894 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:20.901079 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:20.901082 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:20.901083 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:16:20.901163 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:20.901165 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:20.902023 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:20.902025 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:20.903409 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:20.903415 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:20.904160 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:20.904161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:20.904254 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:20.904270 (+ 16us) client_negotiation.cc:769] Sending connection context
0811 05:16:20.904351 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 05:16:20.904423 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3548,"thread_start_us":93,"threads_started":1}
I20260811 05:16:20.904558 25936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:20.897436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45016 (local address 127.28.143.129:43851)
0811 05:16:20.900781 (+ 3345us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:20.900786 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:20.900863 (+ 77us) server_negotiation.cc:407] Connection header received
0811 05:16:20.900973 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:20.900975 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:20.900989 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:20.901062 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:16:20.901231 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:20.901942 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:20.903548 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:20.904076 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:20.904150 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:20.904195 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:20.904446 (+ 251us) server_negotiation.cc:299] Negotiation successful
0811 05:16:20.904494 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3307,"thread_start_us":109,"threads_started":1}
I20260811 05:16:20.905287 25801 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f45eff0b7bf4b41b603307b056177d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:16:00 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:45016
I20260811 05:16:20.905869 25937 tablet_replica.cc:330] stopping tablet replica
I20260811 05:16:20.906001 25937 raft_consensus.cc:2229] T 8f45eff0b7bf4b41b603307b056177d3 P 95bb8596089e4e69ba1427ac129f4a0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:16:20.906078 25937 raft_consensus.cc:2258] T 8f45eff0b7bf4b41b603307b056177d3 P 95bb8596089e4e69ba1427ac129f4a0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:16:20.906432 25937 ts_tablet_manager.cc:1905] T 8f45eff0b7bf4b41b603307b056177d3 P 95bb8596089e4e69ba1427ac129f4a0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:16:20.909354 25937 ts_tablet_manager.cc:1918] T 8f45eff0b7bf4b41b603307b056177d3 P 95bb8596089e4e69ba1427ac129f4a0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:16:20.909449 25937 log.cc:1196] T 8f45eff0b7bf4b41b603307b056177d3 P 95bb8596089e4e69ba1427ac129f4a0f: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/wal/wals/8f45eff0b7bf4b41b603307b056177d3
I20260811 05:16:20.909678 25937 ts_tablet_manager.cc:1939] T 8f45eff0b7bf4b41b603307b056177d3 P 95bb8596089e4e69ba1427ac129f4a0f: Deleting consensus metadata
I20260811 05:16:20.910336 25551 catalog_manager.cc:4841] TS 95bb8596089e4e69ba1427ac129f4a0f (127.28.143.129:43851): tablet 8f45eff0b7bf4b41b603307b056177d3 (table test-table [id=d1ce7da03b3f44e4941a1d6fab7d38bf]) successfully deleted
W20260811 05:16:20.979422 25930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:16:20.979713 25930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:16:20.979755 25930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:16:20.981071 25930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:16:20.981125 25930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:16:20.981153 25930 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.130
I20260811 05:16:20.982656 25930 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:42519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.130:40211
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.143.130
--webserver_port=43843
--tserver_master_addrs=127.28.143.190:36473
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25930
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.130
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:16:20.982913 25930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:16:20.983139 25930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:16:20.988206 25945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:20.988206 25947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:20.988432 25953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:20.988530 25930 server_base.cc:1034] running on GCE node
I20260811 05:16:20.988844 25930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:16:20.989130 25930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:16:20.990288 25930 hybrid_clock.cc:648] HybridClock initialized: now 1786425380990272 us; error 29 us; skew 500 ppm
Aug 11 05:16:20 dist-test-slave-f7th krb5kdc[24572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425380, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:16:20.993108 25930 init.cc:377] Logged in from keytab as kudu/127.28.143.130@KRBTEST.COM (short username kudu)
I20260811 05:16:20.993594 25930 webserver.cc:460] Webserver started at http://127.28.143.130:43843/ using document root <none> and password file <none>
I20260811 05:16:20.993835 25930 fs_manager.cc:359] Metadata directory not provided
I20260811 05:16:20.993914 25930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:16:20.995386 25930 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:16:20.996129 25964 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:20.996402 25930 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:16:20.996529 25930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/wal
uuid: "b9b4e2cafa624126b2b8f0f05cedde73"
format_stamp: "Formatted at 2026-08-11 05:15:59 on dist-test-slave-f7th"
server_key: "4c0c969e5ad3c05464bb43e9174b0e4a"
server_key_iv: "5ce0e781295116ec92b69284b185224f"
server_key_version: "encryptionkey@0"
I20260811 05:16:20.996933 25930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:16:21.014586 25930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:16:21.017012 25930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:16:21.017158 25930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:16:21.017387 25930 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:16:21.017712 25930 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:16:21.017750 25930 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:21.017781 25930 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:16:21.017804 25930 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:21.023659 25930 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.130:40211
I20260811 05:16:21.023721 26136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.130:40211 every 8 connection(s)
I20260811 05:16:21.024093 25930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
Aug 11 05:16:21 dist-test-slave-f7th krb5kdc[24572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425380, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:16:21.033537 25926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:21.024888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:39019 (local address 127.28.143.190:36473)
0811 05:16:21.024943 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:21.024945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:21.025161 (+ 216us) server_negotiation.cc:407] Connection header received
0811 05:16:21.025679 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:21.025681 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:21.025697 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:21.025751 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:21.026726 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:21.027254 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:21.028287 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:21.028509 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:21.028566 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:21.030899 (+ 2333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:21.030902 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:21.030902 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:21.030910 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:21.032165 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:21.032564 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:21.032565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:21.032565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:21.032645 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:21.032866 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:21.032867 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:21.032867 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:21.032991 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:21.033049 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:21.033323 (+ 274us) server_negotiation.cc:299] Negotiation successful
0811 05:16:21.033438 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 05:16:21.033702 26143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:21.024857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.130:39019)
0811 05:16:21.025037 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:21.025076 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:21.025571 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:21.025826 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:21.025830 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:21.026418 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:16:21.026618 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:21.026624 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:21.027359 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:21.027361 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:21.028184 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:21.028187 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:21.028590 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:21.028591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:21.028747 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:21.029264 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:16:21.029273 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:16:21.030792 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:16:21.032247 (+ 1455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:21.032255 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:21.032258 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:21.032491 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:21.032696 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:21.032697 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:21.032697 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:21.032807 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:21.033046 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:16:21.033053 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:16:21.033201 (+ 148us) client_negotiation.cc:769] Sending connection context
0811 05:16:21.033292 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 05:16:21.033465 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":107,"threads_started":1}
I20260811 05:16:21.033758 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 25930
I20260811 05:16:21.034061 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.131:44175
--local_ip_for_outbound_sockets=127.28.143.131
--tserver_master_addrs=127.28.143.190:36473
--webserver_port=37981
--webserver_interface=127.28.143.131
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:42519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:16:21.034168 26137 heartbeater.cc:347] Connected to a master server at 127.28.143.190:36473
I20260811 05:16:21.034286 26137 heartbeater.cc:464] Registering TS with master...
I20260811 05:16:21.034534 26137 heartbeater.cc:511] Master 127.28.143.190:36473 requested a full tablet report, sending...
I20260811 05:16:21.035049 25578 ts_manager.cc:194] Registered new tserver with Master: b9b4e2cafa624126b2b8f0f05cedde73 (127.28.143.130:40211)
I20260811 05:16:21.035452 25578 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.130@KRBTEST.COM'} at 127.28.143.130:39019
W20260811 05:16:21.121511 26145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:16:21.121744 26145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:16:21.121769 26145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:16:21.122907 26145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:16:21.122946 26145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:16:21.122970 26145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.131
I20260811 05:16:21.124464 26145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:42519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.131:44175
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.143.131
--webserver_port=37981
--tserver_master_addrs=127.28.143.190:36473
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.131
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:16:21.124723 26145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:16:21.124930 26145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:16:21.130105 26156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:21.130236 26145 server_base.cc:1034] running on GCE node
W20260811 05:16:21.130148 26153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:21.130119 26154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:21.130532 26145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:16:21.130784 26145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:16:21.131963 26145 hybrid_clock.cc:648] HybridClock initialized: now 1786425381131943 us; error 27 us; skew 500 ppm
Aug 11 05:16:21 dist-test-slave-f7th krb5kdc[24572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425381, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:16:21.134615 26145 init.cc:377] Logged in from keytab as kudu/127.28.143.131@KRBTEST.COM (short username kudu)
I20260811 05:16:21.135128 26145 webserver.cc:460] Webserver started at http://127.28.143.131:37981/ using document root <none> and password file <none>
I20260811 05:16:21.135361 26145 fs_manager.cc:359] Metadata directory not provided
I20260811 05:16:21.135430 26145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:16:21.136945 26145 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 05:16:21.137591 26168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:21.137764 26145 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 05:16:21.137869 26145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/wal
uuid: "169da5b1aeea4a0a88c8ccab57cc2f14"
format_stamp: "Formatted at 2026-08-11 05:16:00 on dist-test-slave-f7th"
server_key: "cb97f7628eb515ea0b7ef627d81e2d18"
server_key_iv: "c5767a466c0e581804665a71927a559c"
server_key_version: "encryptionkey@0"
I20260811 05:16:21.138275 26145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:16:21.150767 26145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:16:21.152832 26145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:16:21.152992 26145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:16:21.153237 26145 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:16:21.153584 26145 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:16:21.153620 26145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:21.153650 26145 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:16:21.153695 26145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:21.159785 26145 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.131:44175
I20260811 05:16:21.159843 26364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.131:44175 every 8 connection(s)
I20260811 05:16:21.160194 26145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
I20260811 05:16:21.160768 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 26145
Aug 11 05:16:21 dist-test-slave-f7th krb5kdc[24572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425381, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:16:21.168718 26370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:21.161117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40970 (local address 127.28.143.190:36473)
0811 05:16:21.161843 (+ 726us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:21.161845 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:21.161853 (+ 8us) server_negotiation.cc:407] Connection header received
0811 05:16:21.161873 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:21.161873 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:21.161883 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:21.161932 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:21.162707 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:21.163221 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:21.164141 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:21.164444 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:21.164525 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:21.166068 (+ 1543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:21.166069 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:21.166069 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:21.166076 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:21.167524 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:21.167914 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:21.167915 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:21.167916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:21.167987 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:21.168166 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:21.168167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:21.168167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:21.168286 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:21.168471 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:21.168508 (+ 37us) server_negotiation.cc:299] Negotiation successful
0811 05:16:21.168591 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":708,"thread_start_us":62,"threads_started":1}
I20260811 05:16:21.170363 25926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:21.160997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:40693 (local address 127.28.143.190:36473)
0811 05:16:21.161055 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:21.161058 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:21.161301 (+ 243us) server_negotiation.cc:407] Connection header received
0811 05:16:21.161785 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:21.161786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:21.161796 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:21.161847 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:21.163065 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:21.163679 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:21.164971 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:21.165230 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:21.165285 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:21.167633 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:21.167634 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:21.167634 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:21.167640 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:21.168906 (+ 1266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:21.169365 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:21.169366 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:21.169367 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:21.169432 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:21.169709 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:21.169710 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:21.169710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:21.169829 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:21.169889 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:21.170167 (+ 278us) server_negotiation.cc:299] Negotiation successful
0811 05:16:21.170265 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 05:16:21.170548 26369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:21.161016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.131:40693)
0811 05:16:21.161178 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:21.161221 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:21.161697 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:21.161966 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:21.161970 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:21.162779 (+ 809us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:16:21.162974 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:21.162979 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:21.163796 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:21.163798 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:21.164894 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:21.164897 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:21.165326 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:21.165327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:21.165480 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:21.165922 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:16:21.165933 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:16:21.167533 (+ 1600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:16:21.168979 (+ 1446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:21.168988 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:21.168991 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:21.169232 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:21.169494 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:21.169495 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:21.169495 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:21.169639 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:21.169888 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:16:21.169896 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:16:21.170043 (+ 147us) client_negotiation.cc:769] Sending connection context
0811 05:16:21.170138 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 05:16:21.170312 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":89,"threads_started":1}
I20260811 05:16:21.171068 26365 heartbeater.cc:347] Connected to a master server at 127.28.143.190:36473
I20260811 05:16:21.171162 26365 heartbeater.cc:464] Registering TS with master...
I20260811 05:16:21.171391 26365 heartbeater.cc:511] Master 127.28.143.190:36473 requested a full tablet report, sending...
I20260811 05:16:21.171763 25578 ts_manager.cc:194] Registered new tserver with Master: 169da5b1aeea4a0a88c8ccab57cc2f14 (127.28.143.131:44175)
I20260811 05:16:21.172122 25578 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.131@KRBTEST.COM'} at 127.28.143.131:40693
I20260811 05:16:21.180866 29246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:16:21.183921 25926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:21.181293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40976 (local address 127.28.143.190:36473)
0811 05:16:21.181352 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:21.181354 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:21.181465 (+ 111us) server_negotiation.cc:407] Connection header received
0811 05:16:21.181527 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:21.181528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:21.181571 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:21.181638 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:16:21.181896 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:21.182389 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:21.183228 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:21.183414 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:21.183490 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:21.183679 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:16:21.183735 (+ 56us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 05:16:21.183861 (+ 126us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40976: token expired
Metrics: {"server-negotiator.queue_time_us":31}
W20260811 05:16:21.184002 25926 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40976: token expired
W20260811 05:16:21.184267 25450 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.28.143.190:36473)
I20260811 05:16:21.190502 25926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:21.184592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40980 (local address 127.28.143.190:36473)
0811 05:16:21.184689 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:21.184691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:21.184698 (+ 7us) server_negotiation.cc:407] Connection header received
0811 05:16:21.184716 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:21.184717 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:21.184724 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:21.184768 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:21.185280 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:21.185741 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:21.186442 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:21.186630 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:21.186684 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:21.187994 (+ 1310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:21.187995 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:21.187995 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:21.188002 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:21.189358 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:21.189719 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:21.189721 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:21.189721 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:21.189776 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:21.189955 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:21.189956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:21.189956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:21.190120 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:21.190216 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:21.190327 (+ 111us) server_negotiation.cc:299] Negotiation successful
0811 05:16:21.190406 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 05:16:21.191848 25578 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40980:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:16:21.193696 25578 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:16:21.195897 25801 tablet_service.cc:1467] Processing CreateTablet for tablet dbacf6b7329f4252a8cf0deadfd18240 (DEFAULT_TABLE table=test-table [id=34da82df6106427c8df577f6dd972968]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:16:21.196142 25801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbacf6b7329f4252a8cf0deadfd18240. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:16:21.197760 25731 tablet_bootstrap.cc:492] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f: Bootstrap starting.
I20260811 05:16:21.198132 25731 tablet_bootstrap.cc:654] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f: Neither blocks nor log segments found. Creating new log.
I20260811 05:16:21.198764 25731 tablet_bootstrap.cc:492] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f: No bootstrap required, opened a new log
I20260811 05:16:21.198818 25731 ts_tablet_manager.cc:1397] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:16:21.198930 25731 raft_consensus.cc:348] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9b4e2cafa624126b2b8f0f05cedde73" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 40211 } } peers { permanent_uuid: "169da5b1aeea4a0a88c8ccab57cc2f14" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44175 } } peers { permanent_uuid: "95bb8596089e4e69ba1427ac129f4a0f" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43851 } }
I20260811 05:16:21.199018 25731 raft_consensus.cc:374] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:16:21.199065 25731 raft_consensus.cc:729] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95bb8596089e4e69ba1427ac129f4a0f, State: Initialized, Role: FOLLOWER
I20260811 05:16:21.199115 25731 consensus_queue.cc:260] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9b4e2cafa624126b2b8f0f05cedde73" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 40211 } } peers { permanent_uuid: "169da5b1aeea4a0a88c8ccab57cc2f14" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44175 } } peers { permanent_uuid: "95bb8596089e4e69ba1427ac129f4a0f" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43851 } }
I20260811 05:16:21.199208 25731 ts_tablet_manager.cc:1428] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:21.199585 25933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:21.195666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:40211 (local address 127.0.0.1:35948)
0811 05:16:21.195706 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:21.195718 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:21.195756 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:21.196213 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:21.196214 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:21.196216 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:16:21.196269 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:21.196270 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:21.197489 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:21.197490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:21.198420 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:21.198424 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:21.199360 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:21.199362 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:21.199424 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:21.199429 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 05:16:21.199458 (+ 29us) client_negotiation.cc:240] Negotiation successful
0811 05:16:21.199512 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260811 05:16:21.199913 26373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:21.195800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35948 (local address 127.28.143.130:40211)
0811 05:16:21.196018 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:21.196023 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:21.196035 (+ 12us) server_negotiation.cc:407] Connection header received
0811 05:16:21.196079 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:21.196081 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:21.196097 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:21.196337 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:16:21.196432 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:21.197413 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:21.198503 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:21.199256 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:21.199703 (+ 447us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:21.199779 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:21.199808 (+ 29us) server_negotiation.cc:299] Negotiation successful
0811 05:16:21.199850 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":97,"threads_started":1}
I20260811 05:16:21.200223 26376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:21.196103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59990 (local address 127.28.143.131:44175)
0811 05:16:21.196386 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:21.196390 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:21.196399 (+ 9us) server_negotiation.cc:407] Connection header received
0811 05:16:21.196436 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:21.196438 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:21.196454 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:21.196544 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:16:21.196795 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:21.197951 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:21.199172 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:21.199847 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:21.199912 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:21.199966 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:21.200080 (+ 114us) server_negotiation.cc:299] Negotiation successful
0811 05:16:21.200145 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":88,"threads_started":1}
I20260811 05:16:21.200421 26374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:21.195755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.131:44175 (local address 127.0.0.1:59990)
0811 05:16:21.196088 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:21.196100 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:21.196157 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:21.196568 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:21.196570 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:21.196572 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:16:21.196643 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:21.196645 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:21.198025 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:21.198026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:21.199077 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:21.199080 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:21.199920 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:21.199921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:21.199986 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:21.199990 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:16:21.200308 (+ 318us) client_negotiation.cc:240] Negotiation successful
0811 05:16:21.200366 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":173,"threads_started":1}
I20260811 05:16:21.200704 26037 tablet_service.cc:1467] Processing CreateTablet for tablet dbacf6b7329f4252a8cf0deadfd18240 (DEFAULT_TABLE table=test-table [id=34da82df6106427c8df577f6dd972968]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:16:21.201048 26037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbacf6b7329f4252a8cf0deadfd18240. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:16:21.203790 26378 tablet_bootstrap.cc:492] T dbacf6b7329f4252a8cf0deadfd18240 P b9b4e2cafa624126b2b8f0f05cedde73: Bootstrap starting.
I20260811 05:16:21.204396 26261 tablet_service.cc:1467] Processing CreateTablet for tablet dbacf6b7329f4252a8cf0deadfd18240 (DEFAULT_TABLE table=test-table [id=34da82df6106427c8df577f6dd972968]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:16:21.204722 26378 tablet_bootstrap.cc:654] T dbacf6b7329f4252a8cf0deadfd18240 P b9b4e2cafa624126b2b8f0f05cedde73: Neither blocks nor log segments found. Creating new log.
I20260811 05:16:21.204739 26261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbacf6b7329f4252a8cf0deadfd18240. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:16:21.205233 26378 log.cc:824] T dbacf6b7329f4252a8cf0deadfd18240 P b9b4e2cafa624126b2b8f0f05cedde73: Log is configured to *not* fsync() on all Append() calls
I20260811 05:16:21.205999 26378 tablet_bootstrap.cc:492] T dbacf6b7329f4252a8cf0deadfd18240 P b9b4e2cafa624126b2b8f0f05cedde73: No bootstrap required, opened a new log
I20260811 05:16:21.206058 26378 ts_tablet_manager.cc:1397] T dbacf6b7329f4252a8cf0deadfd18240 P b9b4e2cafa624126b2b8f0f05cedde73: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 05:16:21.207441 26378 raft_consensus.cc:348] T dbacf6b7329f4252a8cf0deadfd18240 P b9b4e2cafa624126b2b8f0f05cedde73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9b4e2cafa624126b2b8f0f05cedde73" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 40211 } } peers { permanent_uuid: "169da5b1aeea4a0a88c8ccab57cc2f14" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44175 } } peers { permanent_uuid: "95bb8596089e4e69ba1427ac129f4a0f" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43851 } }
I20260811 05:16:21.207717 26378 raft_consensus.cc:374] T dbacf6b7329f4252a8cf0deadfd18240 P b9b4e2cafa624126b2b8f0f05cedde73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:16:21.207715 26381 tablet_bootstrap.cc:492] T dbacf6b7329f4252a8cf0deadfd18240 P 169da5b1aeea4a0a88c8ccab57cc2f14: Bootstrap starting.
I20260811 05:16:21.207834 26378 raft_consensus.cc:729] T dbacf6b7329f4252a8cf0deadfd18240 P b9b4e2cafa624126b2b8f0f05cedde73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9b4e2cafa624126b2b8f0f05cedde73, State: Initialized, Role: FOLLOWER
I20260811 05:16:21.207989 26378 consensus_queue.cc:260] T dbacf6b7329f4252a8cf0deadfd18240 P b9b4e2cafa624126b2b8f0f05cedde73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9b4e2cafa624126b2b8f0f05cedde73" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 40211 } } peers { permanent_uuid: "169da5b1aeea4a0a88c8ccab57cc2f14" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44175 } } peers { permanent_uuid: "95bb8596089e4e69ba1427ac129f4a0f" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43851 } }
I20260811 05:16:21.208266 26378 ts_tablet_manager.cc:1428] T dbacf6b7329f4252a8cf0deadfd18240 P b9b4e2cafa624126b2b8f0f05cedde73: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 05:16:21.208321 26137 heartbeater.cc:503] Master 127.28.143.190:36473 was elected leader, sending a full tablet report...
I20260811 05:16:21.208433 26381 tablet_bootstrap.cc:654] T dbacf6b7329f4252a8cf0deadfd18240 P 169da5b1aeea4a0a88c8ccab57cc2f14: Neither blocks nor log segments found. Creating new log.
I20260811 05:16:21.209160 26381 log.cc:824] T dbacf6b7329f4252a8cf0deadfd18240 P 169da5b1aeea4a0a88c8ccab57cc2f14: Log is configured to *not* fsync() on all Append() calls
I20260811 05:16:21.209995 26381 tablet_bootstrap.cc:492] T dbacf6b7329f4252a8cf0deadfd18240 P 169da5b1aeea4a0a88c8ccab57cc2f14: No bootstrap required, opened a new log
I20260811 05:16:21.210067 26381 ts_tablet_manager.cc:1397] T dbacf6b7329f4252a8cf0deadfd18240 P 169da5b1aeea4a0a88c8ccab57cc2f14: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260811 05:16:21.211470 26381 raft_consensus.cc:348] T dbacf6b7329f4252a8cf0deadfd18240 P 169da5b1aeea4a0a88c8ccab57cc2f14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9b4e2cafa624126b2b8f0f05cedde73" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 40211 } } peers { permanent_uuid: "169da5b1aeea4a0a88c8ccab57cc2f14" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44175 } } peers { permanent_uuid: "95bb8596089e4e69ba1427ac129f4a0f" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43851 } }
I20260811 05:16:21.211622 26381 raft_consensus.cc:374] T dbacf6b7329f4252a8cf0deadfd18240 P 169da5b1aeea4a0a88c8ccab57cc2f14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:16:21.211654 26381 raft_consensus.cc:729] T dbacf6b7329f4252a8cf0deadfd18240 P 169da5b1aeea4a0a88c8ccab57cc2f14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 169da5b1aeea4a0a88c8ccab57cc2f14, State: Initialized, Role: FOLLOWER
I20260811 05:16:21.211781 26381 consensus_queue.cc:260] T dbacf6b7329f4252a8cf0deadfd18240 P 169da5b1aeea4a0a88c8ccab57cc2f14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9b4e2cafa624126b2b8f0f05cedde73" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 40211 } } peers { permanent_uuid: "169da5b1aeea4a0a88c8ccab57cc2f14" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44175 } } peers { permanent_uuid: "95bb8596089e4e69ba1427ac129f4a0f" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43851 } }
I20260811 05:16:21.212138 26381 ts_tablet_manager.cc:1428] T dbacf6b7329f4252a8cf0deadfd18240 P 169da5b1aeea4a0a88c8ccab57cc2f14: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 05:16:21.212230 26365 heartbeater.cc:503] Master 127.28.143.190:36473 was elected leader, sending a full tablet report...
I20260811 05:16:21.270375 26386 raft_consensus.cc:482] T dbacf6b7329f4252a8cf0deadfd18240 P 169da5b1aeea4a0a88c8ccab57cc2f14 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:16:21.270529 26386 raft_consensus.cc:504] T dbacf6b7329f4252a8cf0deadfd18240 P 169da5b1aeea4a0a88c8ccab57cc2f14 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9b4e2cafa624126b2b8f0f05cedde73" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 40211 } } peers { permanent_uuid: "169da5b1aeea4a0a88c8ccab57cc2f14" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44175 } } peers { permanent_uuid: "95bb8596089e4e69ba1427ac129f4a0f" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43851 } }
I20260811 05:16:21.270915 26386 leader_election.cc:288] T dbacf6b7329f4252a8cf0deadfd18240 P 169da5b1aeea4a0a88c8ccab57cc2f14 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b9b4e2cafa624126b2b8f0f05cedde73 (127.28.143.130:40211), 95bb8596089e4e69ba1427ac129f4a0f (127.28.143.129:43851)
I20260811 05:16:21.273772 25936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:21.271155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:48253 (local address 127.28.143.129:43851)
0811 05:16:21.271209 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:21.271211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:21.271222 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:16:21.271254 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:21.271255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:21.271265 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:21.271346 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:16:21.271505 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:21.272066 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:21.273081 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:21.273452 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:21.273514 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:21.273537 (+ 23us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:21.273670 (+ 133us) server_negotiation.cc:299] Negotiation successful
0811 05:16:21.273707 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 05:16:21.273772 26373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:21.271155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:36547 (local address 127.28.143.130:40211)
0811 05:16:21.271213 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:21.271215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:21.271407 (+ 192us) server_negotiation.cc:407] Connection header received
0811 05:16:21.271439 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:21.271440 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:21.271451 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:21.271501 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:16:21.271640 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:21.272094 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:21.273085 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:21.273459 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:21.273524 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:21.273534 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:21.273663 (+ 129us) server_negotiation.cc:299] Negotiation successful
0811 05:16:21.273697 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 05:16:21.273808 26369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:21.271077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:43851 (local address 127.28.143.131:48253)
0811 05:16:21.271140 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:21.271151 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:21.271194 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:21.271350 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:21.271352 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:21.271353 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:16:21.271434 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:21.271436 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:21.272147 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:21.272148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:21.272994 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:21.272996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:21.273524 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:21.273525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:21.273609 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:21.273612 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 05:16:21.273659 (+ 47us) client_negotiation.cc:240] Negotiation successful
0811 05:16:21.273699 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 05:16:21.273830 26387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:21.271087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:40211 (local address 127.28.143.131:36547)
0811 05:16:21.271284 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:21.271295 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:21.271355 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:21.271509 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:21.271510 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:21.271512 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:16:21.271576 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:21.271578 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:21.272172 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:21.272172 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:21.272998 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:21.273000 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:21.273535 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:21.273536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:21.273600 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:21.273604 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:16:21.273650 (+ 46us) client_negotiation.cc:240] Negotiation successful
0811 05:16:21.273690 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":19,"thread_start_us":103,"threads_started":1}
I20260811 05:16:21.274216 25841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbacf6b7329f4252a8cf0deadfd18240" candidate_uuid: "169da5b1aeea4a0a88c8ccab57cc2f14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95bb8596089e4e69ba1427ac129f4a0f" is_pre_election: true
I20260811 05:16:21.274264 26073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbacf6b7329f4252a8cf0deadfd18240" candidate_uuid: "169da5b1aeea4a0a88c8ccab57cc2f14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9b4e2cafa624126b2b8f0f05cedde73" is_pre_election: true
I20260811 05:16:21.274395 25841 raft_consensus.cc:2454] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 169da5b1aeea4a0a88c8ccab57cc2f14 in term 0.
I20260811 05:16:21.274420 26073 raft_consensus.cc:2454] T dbacf6b7329f4252a8cf0deadfd18240 P b9b4e2cafa624126b2b8f0f05cedde73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 169da5b1aeea4a0a88c8ccab57cc2f14 in term 0.
W20260811 05:16:21.274601 26139 tablet.cc:2369] T dbacf6b7329f4252a8cf0deadfd18240 P b9b4e2cafa624126b2b8f0f05cedde73: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:16:21.274645 26171 leader_election.cc:302] T dbacf6b7329f4252a8cf0deadfd18240 P 169da5b1aeea4a0a88c8ccab57cc2f14 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 169da5b1aeea4a0a88c8ccab57cc2f14, b9b4e2cafa624126b2b8f0f05cedde73; no voters:
I20260811 05:16:21.274803 26386 raft_consensus.cc:2781] T dbacf6b7329f4252a8cf0deadfd18240 P 169da5b1aeea4a0a88c8ccab57cc2f14 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:16:21.274853 26386 raft_consensus.cc:482] T dbacf6b7329f4252a8cf0deadfd18240 P 169da5b1aeea4a0a88c8ccab57cc2f14 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:16:21.274895 26386 raft_consensus.cc:3037] T dbacf6b7329f4252a8cf0deadfd18240 P 169da5b1aeea4a0a88c8ccab57cc2f14 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:16:21.275635 26386 raft_consensus.cc:504] T dbacf6b7329f4252a8cf0deadfd18240 P 169da5b1aeea4a0a88c8ccab57cc2f14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9b4e2cafa624126b2b8f0f05cedde73" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 40211 } } peers { permanent_uuid: "169da5b1aeea4a0a88c8ccab57cc2f14" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44175 } } peers { permanent_uuid: "95bb8596089e4e69ba1427ac129f4a0f" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43851 } }
I20260811 05:16:21.275761 26386 leader_election.cc:288] T dbacf6b7329f4252a8cf0deadfd18240 P 169da5b1aeea4a0a88c8ccab57cc2f14 [CANDIDATE]: Term 1 election: Requested vote from peers b9b4e2cafa624126b2b8f0f05cedde73 (127.28.143.130:40211), 95bb8596089e4e69ba1427ac129f4a0f (127.28.143.129:43851)
I20260811 05:16:21.275970 25841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbacf6b7329f4252a8cf0deadfd18240" candidate_uuid: "169da5b1aeea4a0a88c8ccab57cc2f14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95bb8596089e4e69ba1427ac129f4a0f"
I20260811 05:16:21.275986 26073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbacf6b7329f4252a8cf0deadfd18240" candidate_uuid: "169da5b1aeea4a0a88c8ccab57cc2f14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9b4e2cafa624126b2b8f0f05cedde73"
I20260811 05:16:21.276053 26073 raft_consensus.cc:3037] T dbacf6b7329f4252a8cf0deadfd18240 P b9b4e2cafa624126b2b8f0f05cedde73 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:16:21.276059 25841 raft_consensus.cc:3037] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:16:21.276829 26073 raft_consensus.cc:2454] T dbacf6b7329f4252a8cf0deadfd18240 P b9b4e2cafa624126b2b8f0f05cedde73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 169da5b1aeea4a0a88c8ccab57cc2f14 in term 1.
I20260811 05:16:21.277022 26171 leader_election.cc:302] T dbacf6b7329f4252a8cf0deadfd18240 P 169da5b1aeea4a0a88c8ccab57cc2f14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 169da5b1aeea4a0a88c8ccab57cc2f14, b9b4e2cafa624126b2b8f0f05cedde73; no voters:
I20260811 05:16:21.277168 26386 raft_consensus.cc:2781] T dbacf6b7329f4252a8cf0deadfd18240 P 169da5b1aeea4a0a88c8ccab57cc2f14 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:16:21.277333 25841 raft_consensus.cc:2454] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 169da5b1aeea4a0a88c8ccab57cc2f14 in term 1.
I20260811 05:16:21.277345 26386 raft_consensus.cc:686] T dbacf6b7329f4252a8cf0deadfd18240 P 169da5b1aeea4a0a88c8ccab57cc2f14 [term 1 LEADER]: Becoming Leader. State: Replica: 169da5b1aeea4a0a88c8ccab57cc2f14, State: Running, Role: LEADER
I20260811 05:16:21.277563 26386 consensus_queue.cc:237] T dbacf6b7329f4252a8cf0deadfd18240 P 169da5b1aeea4a0a88c8ccab57cc2f14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9b4e2cafa624126b2b8f0f05cedde73" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 40211 } } peers { permanent_uuid: "169da5b1aeea4a0a88c8ccab57cc2f14" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44175 } } peers { permanent_uuid: "95bb8596089e4e69ba1427ac129f4a0f" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43851 } }
I20260811 05:16:21.278606 25578 catalog_manager.cc:5495] T dbacf6b7329f4252a8cf0deadfd18240 P 169da5b1aeea4a0a88c8ccab57cc2f14 reported cstate change: term changed from 0 to 1, leader changed from <none> to 169da5b1aeea4a0a88c8ccab57cc2f14 (127.28.143.131). New cstate: current_term: 1 leader_uuid: "169da5b1aeea4a0a88c8ccab57cc2f14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9b4e2cafa624126b2b8f0f05cedde73" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 40211 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "169da5b1aeea4a0a88c8ccab57cc2f14" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44175 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95bb8596089e4e69ba1427ac129f4a0f" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43851 } health_report { overall_health: UNKNOWN } } }
I20260811 05:16:21.293532 26376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:21.291170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59998 (local address 127.28.143.131:44175)
0811 05:16:21.291235 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:21.291237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:21.291245 (+ 8us) server_negotiation.cc:407] Connection header received
0811 05:16:21.291287 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:21.291288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:21.291307 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:21.291366 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:16:21.291491 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:21.292001 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:21.292777 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:21.292982 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:21.293038 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:21.293181 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:16:21.293292 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:16:21.293367 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:21.293406 (+ 39us) server_negotiation.cc:299] Negotiation successful
0811 05:16:21.293426 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 05:16:21.295578 26073 raft_consensus.cc:1261] T dbacf6b7329f4252a8cf0deadfd18240 P b9b4e2cafa624126b2b8f0f05cedde73 [term 1 FOLLOWER]: Refusing update from remote peer 169da5b1aeea4a0a88c8ccab57cc2f14: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:16:21.295636 25841 raft_consensus.cc:1261] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [term 1 FOLLOWER]: Refusing update from remote peer 169da5b1aeea4a0a88c8ccab57cc2f14: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:16:21.295965 26386 consensus_queue.cc:1035] T dbacf6b7329f4252a8cf0deadfd18240 P 169da5b1aeea4a0a88c8ccab57cc2f14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9b4e2cafa624126b2b8f0f05cedde73" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 40211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:16:21.296114 26388 consensus_queue.cc:1035] T dbacf6b7329f4252a8cf0deadfd18240 P 169da5b1aeea4a0a88c8ccab57cc2f14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95bb8596089e4e69ba1427ac129f4a0f" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:16:21.300973 25577 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40980:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:16:21.301076 25577 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40980:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:16:21.301828 25577 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115: Sending DeleteTablet for 3 replicas of tablet dbacf6b7329f4252a8cf0deadfd18240
I20260811 05:16:21.302124 25801 tablet_service.cc:1514] Processing DeleteTablet for tablet dbacf6b7329f4252a8cf0deadfd18240 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:16:21 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:45016
I20260811 05:16:21.302132 26037 tablet_service.cc:1514] Processing DeleteTablet for tablet dbacf6b7329f4252a8cf0deadfd18240 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:16:21 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:35948
I20260811 05:16:21.302258 26261 tablet_service.cc:1514] Processing DeleteTablet for tablet dbacf6b7329f4252a8cf0deadfd18240 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:16:21 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:59990
I20260811 05:16:21.302333 25937 tablet_replica.cc:330] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f: stopping tablet replica
I20260811 05:16:21.302404 25937 raft_consensus.cc:2229] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:16:21.302456 25937 raft_consensus.cc:2258] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:16:21.302469 26412 tablet_replica.cc:330] T dbacf6b7329f4252a8cf0deadfd18240 P 169da5b1aeea4a0a88c8ccab57cc2f14: stopping tablet replica
I20260811 05:16:21.302553 26412 raft_consensus.cc:2229] T dbacf6b7329f4252a8cf0deadfd18240 P 169da5b1aeea4a0a88c8ccab57cc2f14 [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:16:21.302655 26412 raft_consensus.cc:2258] T dbacf6b7329f4252a8cf0deadfd18240 P 169da5b1aeea4a0a88c8ccab57cc2f14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:16:21.302655 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 25653
I20260811 05:16:21.334635 26412 ts_tablet_manager.cc:1905] T dbacf6b7329f4252a8cf0deadfd18240 P 169da5b1aeea4a0a88c8ccab57cc2f14: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:16:21.336701 26412 ts_tablet_manager.cc:1918] T dbacf6b7329f4252a8cf0deadfd18240 P 169da5b1aeea4a0a88c8ccab57cc2f14: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:16:21.336903 26412 log.cc:1196] T dbacf6b7329f4252a8cf0deadfd18240 P 169da5b1aeea4a0a88c8ccab57cc2f14: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/wal/wals/dbacf6b7329f4252a8cf0deadfd18240
I20260811 05:16:21.337232 26412 ts_tablet_manager.cc:1939] T dbacf6b7329f4252a8cf0deadfd18240 P 169da5b1aeea4a0a88c8ccab57cc2f14: Deleting consensus metadata
I20260811 05:16:21.343096 25555 catalog_manager.cc:4841] TS 169da5b1aeea4a0a88c8ccab57cc2f14 (127.28.143.131:44175): tablet dbacf6b7329f4252a8cf0deadfd18240 (table test-table [id=34da82df6106427c8df577f6dd972968]) successfully deleted
I20260811 05:16:21.344870 26411 tablet_replica.cc:330] T dbacf6b7329f4252a8cf0deadfd18240 P b9b4e2cafa624126b2b8f0f05cedde73: stopping tablet replica
I20260811 05:16:21.345059 26411 raft_consensus.cc:2229] T dbacf6b7329f4252a8cf0deadfd18240 P b9b4e2cafa624126b2b8f0f05cedde73 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:16:21.345203 26411 raft_consensus.cc:2258] T dbacf6b7329f4252a8cf0deadfd18240 P b9b4e2cafa624126b2b8f0f05cedde73 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:16:21.345814 26411 ts_tablet_manager.cc:1905] T dbacf6b7329f4252a8cf0deadfd18240 P b9b4e2cafa624126b2b8f0f05cedde73: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 05:16:21.347815 25551 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.143.129:43851 (error 108)
I20260811 05:16:21.348279 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 25930
I20260811 05:16:21.348395 26374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:21.348247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:43851 (local address 127.0.0.1:45028)
0811 05:16:21.348314 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:21.348357 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.129:43851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 05:16:21.360954 25551 catalog_manager.cc:4584] TS 95bb8596089e4e69ba1427ac129f4a0f (127.28.143.129:43851): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dbacf6b7329f4252a8cf0deadfd18240: Network error: Client connection negotiation failed: client connection to 127.28.143.129:43851: connect: Connection refused (error 111)
I20260811 05:16:21.365947 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 26145
I20260811 05:16:21.366084 26374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:21.365940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:40211 (local address 127.0.0.1:35956)
0811 05:16:21.365997 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:21.366037 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.130:40211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260811 05:16:21.414117 26374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:21.413964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:43851 (local address 127.0.0.1:45036)
0811 05:16:21.414034 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:21.414073 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.129:43851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 05:16:21.414582 26374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:21.414483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:43851 (local address 127.0.0.1:45044)
0811 05:16:21.414533 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:21.414562 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.129:43851: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260811 05:16:21.416987 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 25516
I20260811 05:16:41.436506 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:36473
--webserver_interface=127.28.143.190
--webserver_port=38351
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:42519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:36473
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:16:41.527824 26414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:16:41.528075 26414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:16:41.528100 26414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:16:41.529371 26414 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 05:16:41.529429 26414 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 05:16:41.529443 26414 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 05:16:41.529453 26414 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:16:41.529464 26414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:16:41.529477 26414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:16:41.529488 26414 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:16:41.529500 26414 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:16:41.531250 26414 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:42519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:36473
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:36473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=38351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26414
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:16:41.531517 26414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:16:41.531762 26414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:16:41.537352 26425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:41.537331 26422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:41.537382 26414 server_base.cc:1034] running on GCE node
W20260811 05:16:41.537331 26421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:41.537959 26414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:16:41.538283 26414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:16:41.539466 26414 hybrid_clock.cc:648] HybridClock initialized: now 1786425401539444 us; error 37 us; skew 500 ppm
Aug 11 05:16:41 dist-test-slave-f7th krb5kdc[24572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425401, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:16:41.542253 26414 init.cc:377] Logged in from keytab as kudu/127.28.143.190@KRBTEST.COM (short username kudu)
I20260811 05:16:41.542758 26414 webserver.cc:460] Webserver started at http://127.28.143.190:38351/ using document root <none> and password file <none>
I20260811 05:16:41.542980 26414 fs_manager.cc:359] Metadata directory not provided
I20260811 05:16:41.543031 26414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:16:41.544551 26414 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:16:41.545440 26437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:41.545663 26414 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 05:16:41.545770 26414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "763111c9cea64e29a4dc12beb1394115"
format_stamp: "Formatted at 2026-08-11 05:15:59 on dist-test-slave-f7th"
server_key: "6c5bc1fc8a07b2e380406551df49f7a5"
server_key_iv: "366ad4c1680a74560d1671bc30f58186"
server_key_version: "encryptionkey@0"
I20260811 05:16:41.546175 26414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:16:41.556567 26414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:16:41.559371 26414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:16:41.559533 26414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:16:41.564021 26414 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:36473
I20260811 05:16:41.564067 26531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:36473 every 8 connection(s)
I20260811 05:16:41.564478 26414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:16:41.566270 26532 sys_catalog.cc:263] Verifying existing consensus state
I20260811 05:16:41.566987 26532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115: Bootstrap starting.
I20260811 05:16:41.574275 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 26414
I20260811 05:16:41.574586 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.129:43851
--local_ip_for_outbound_sockets=127.28.143.129
--tserver_master_addrs=127.28.143.190:36473
--webserver_port=41097
--webserver_interface=127.28.143.129
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:42519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:16:41.576332 26532 log.cc:824] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115: Log is configured to *not* fsync() on all Append() calls
I20260811 05:16:41.578765 26532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 05:16:41.579082 26532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115: Bootstrap complete.
I20260811 05:16:41.581555 26532 raft_consensus.cc:348] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "763111c9cea64e29a4dc12beb1394115" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 36473 } }
I20260811 05:16:41.581809 26532 raft_consensus.cc:729] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 763111c9cea64e29a4dc12beb1394115, State: Initialized, Role: FOLLOWER
I20260811 05:16:41.581977 26532 consensus_queue.cc:260] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "763111c9cea64e29a4dc12beb1394115" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 36473 } }
I20260811 05:16:41.582069 26532 raft_consensus.cc:388] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:16:41.582124 26532 raft_consensus.cc:482] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:16:41.582175 26532 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [term 2 FOLLOWER]: Advancing to term 3
I20260811 05:16:41.583256 26532 raft_consensus.cc:504] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "763111c9cea64e29a4dc12beb1394115" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 36473 } }
I20260811 05:16:41.583369 26532 leader_election.cc:302] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 763111c9cea64e29a4dc12beb1394115; no voters:
I20260811 05:16:41.583555 26532 leader_election.cc:288] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 05:16:41.583611 26539 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 05:16:41.583773 26539 raft_consensus.cc:686] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [term 3 LEADER]: Becoming Leader. State: Replica: 763111c9cea64e29a4dc12beb1394115, State: Running, Role: LEADER
I20260811 05:16:41.583865 26532 sys_catalog.cc:564] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:16:41.583931 26539 consensus_queue.cc:237] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "763111c9cea64e29a4dc12beb1394115" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 36473 } }
I20260811 05:16:41.584319 26540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "763111c9cea64e29a4dc12beb1394115" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "763111c9cea64e29a4dc12beb1394115" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 36473 } } }
I20260811 05:16:41.584399 26540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [sys.catalog]: This master's current role is: LEADER
I20260811 05:16:41.586488 26541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 763111c9cea64e29a4dc12beb1394115. Latest consensus state: current_term: 3 leader_uuid: "763111c9cea64e29a4dc12beb1394115" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "763111c9cea64e29a4dc12beb1394115" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 36473 } } }
I20260811 05:16:41.586666 26541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115 [sys.catalog]: This master's current role is: LEADER
I20260811 05:16:41.586855 26546 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:16:41.587905 26546 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:16:41.588105 26546 catalog_manager.cc:1259] Loaded cluster ID: 7b5ac7d58ac345428678d172a6e7d067
I20260811 05:16:41.588244 26546 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:16:41.589392 26546 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:16:41.589629 26546 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115: Loaded TSK: 0
I20260811 05:16:41.589673 26546 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 763111c9cea64e29a4dc12beb1394115: Loaded TSK: 1
F20260811 05:16:41.589896 26546 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fae8096c9ad google::LogMessage::Fail() at ??:0
@ 0x7fae8096f88a google::LogMessage::SendToLog() at ??:0
@ 0x7fae8096c4de google::LogMessage::Flush() at ??:0
@ 0x7fae8096da19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fae830d9104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fae830d6d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fae888c61cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fae888c648e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fae888c6594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fae8888cc48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fae88896ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fae8283a0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fae8282f954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fae836936db start_thread at ??:0
@ 0x7fae7f46a61f clone at ??:0
W20260811 05:16:41.687084 26534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:16:41.687301 26534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:16:41.687327 26534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:16:41.688565 26534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:16:41.688647 26534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:16:41.688691 26534 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.129
I20260811 05:16:41.690304 26534 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:42519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.129:43851
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.143.129
--webserver_port=41097
--tserver_master_addrs=127.28.143.190:36473
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.129
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:16:41.690577 26534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:16:41.690820 26534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:16:41.696013 26611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:41.696020 26603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:41.696146 26604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:41.696762 26534 server_base.cc:1034] running on GCE node
I20260811 05:16:41.696995 26534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:16:41.697266 26534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:16:41.698448 26534 hybrid_clock.cc:648] HybridClock initialized: now 1786425401698411 us; error 45 us; skew 500 ppm
Aug 11 05:16:41 dist-test-slave-f7th krb5kdc[24572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425401, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:16:41.704653 26534 init.cc:377] Logged in from keytab as kudu/127.28.143.129@KRBTEST.COM (short username kudu)
I20260811 05:16:41.705317 26534 webserver.cc:460] Webserver started at http://127.28.143.129:41097/ using document root <none> and password file <none>
I20260811 05:16:41.705651 26534 fs_manager.cc:359] Metadata directory not provided
I20260811 05:16:41.705735 26534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:16:41.707655 26534 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 05:16:41.711027 26622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:41.711239 26534 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 05:16:41.711345 26534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/wal
uuid: "95bb8596089e4e69ba1427ac129f4a0f"
format_stamp: "Formatted at 2026-08-11 05:15:59 on dist-test-slave-f7th"
server_key: "808265f319c6dda4abdb4683fab4dd79"
server_key_iv: "4e52b2f22d5584f5985c214383965cbc"
server_key_version: "encryptionkey@0"
I20260811 05:16:41.711717 26534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:16:41.729221 26534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:16:41.732205 26534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:16:41.732368 26534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:16:41.732683 26534 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:16:41.733227 26637 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 05:16:41.735710 26534 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 05:16:41.735787 26534 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260811 05:16:41.735829 26534 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 05:16:41.736644 26534 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 05:16:41.736696 26534 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.001s
I20260811 05:16:41.736785 26637 tablet_bootstrap.cc:492] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f: Bootstrap starting.
I20260811 05:16:41.744760 26534 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.129:43851
I20260811 05:16:41.744825 26828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.129:43851 every 8 connection(s)
I20260811 05:16:41.745268 26534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
I20260811 05:16:41.748134 26637 log.cc:824] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f: Log is configured to *not* fsync() on all Append() calls
I20260811 05:16:41.751379 26637 tablet_bootstrap.cc:492] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 05:16:41.751619 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 26534
I20260811 05:16:41.751766 26637 tablet_bootstrap.cc:492] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f: Bootstrap complete.
I20260811 05:16:41.751914 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.130:40211
--local_ip_for_outbound_sockets=127.28.143.130
--tserver_master_addrs=127.28.143.190:36473
--webserver_port=43843
--webserver_interface=127.28.143.130
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:42519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:16:41.752929 26637 ts_tablet_manager.cc:1397] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.016s
I20260811 05:16:41.754917 26637 raft_consensus.cc:348] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9b4e2cafa624126b2b8f0f05cedde73" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 40211 } } peers { permanent_uuid: "169da5b1aeea4a0a88c8ccab57cc2f14" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44175 } } peers { permanent_uuid: "95bb8596089e4e69ba1427ac129f4a0f" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43851 } }
I20260811 05:16:41.755333 26637 raft_consensus.cc:729] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95bb8596089e4e69ba1427ac129f4a0f, State: Initialized, Role: FOLLOWER
I20260811 05:16:41.755556 26637 consensus_queue.cc:260] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9b4e2cafa624126b2b8f0f05cedde73" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 40211 } } peers { permanent_uuid: "169da5b1aeea4a0a88c8ccab57cc2f14" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44175 } } peers { permanent_uuid: "95bb8596089e4e69ba1427ac129f4a0f" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43851 } }
I20260811 05:16:41.755929 26637 ts_tablet_manager.cc:1428] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
W20260811 05:16:41.805281 26833 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:16:41.746013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.129:39921)
0811 05:16:41.746198 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:41.746247 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:41.746738 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:41.805069 (+ 58331us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":81,"threads_started":1}
W20260811 05:16:41.805642 26830 heartbeater.cc:644] Failed to heartbeat to 127.28.143.190:36473 (0 consecutive failures): Network error: Failed to ping master at 127.28.143.190:36473: Client connection negotiation failed: client connection to 127.28.143.190:36473: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 05:16:41.806059 26833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:41.805901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.129:54295)
0811 05:16:41.805962 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:41.806027 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260811 05:16:41.806476 26833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:41.806378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.129:43317)
0811 05:16:41.806428 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:41.806457 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260811 05:16:41.806648 26830 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 05:16:41.849061 26835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:16:41.849305 26835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:16:41.849335 26835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:16:41.850673 26835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:16:41.850824 26835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:16:41.850916 26835 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.130
I20260811 05:16:41.853053 26835 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:42519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.130:40211
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.143.130
--webserver_port=43843
--tserver_master_addrs=127.28.143.190:36473
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26835
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.130
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:16:41.853327 26835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:16:41.853549 26835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:16:41.858987 26847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:41.858947 26842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:41.858958 26844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:41.859323 26835 server_base.cc:1034] running on GCE node
I20260811 05:16:41.859582 26835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:16:41.859846 26835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:16:41.861016 26835 hybrid_clock.cc:648] HybridClock initialized: now 1786425401860996 us; error 38 us; skew 500 ppm
Aug 11 05:16:41 dist-test-slave-f7th krb5kdc[24572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425401, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:16:41.863857 26835 init.cc:377] Logged in from keytab as kudu/127.28.143.130@KRBTEST.COM (short username kudu)
I20260811 05:16:41.864427 26835 webserver.cc:460] Webserver started at http://127.28.143.130:43843/ using document root <none> and password file <none>
I20260811 05:16:41.864714 26835 fs_manager.cc:359] Metadata directory not provided
I20260811 05:16:41.864771 26835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:16:41.866271 26835 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 05:16:41.867136 26861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:41.867352 26835 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 05:16:41.867465 26835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/wal
uuid: "b9b4e2cafa624126b2b8f0f05cedde73"
format_stamp: "Formatted at 2026-08-11 05:15:59 on dist-test-slave-f7th"
server_key: "4c0c969e5ad3c05464bb43e9174b0e4a"
server_key_iv: "5ce0e781295116ec92b69284b185224f"
server_key_version: "encryptionkey@0"
I20260811 05:16:41.867846 26835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:16:41.891780 26835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:16:41.894282 26835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:16:41.894450 26835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:16:41.894699 26835 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:16:41.895237 26868 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260811 05:16:41.896963 26868 ts_tablet_manager.cc:1852] T dbacf6b7329f4252a8cf0deadfd18240 P b9b4e2cafa624126b2b8f0f05cedde73: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260811 05:16:41.897238 26868 ts_tablet_manager.cc:1905] T dbacf6b7329f4252a8cf0deadfd18240 P b9b4e2cafa624126b2b8f0f05cedde73: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:16:41.899842 26868 ts_tablet_manager.cc:1918] T dbacf6b7329f4252a8cf0deadfd18240 P b9b4e2cafa624126b2b8f0f05cedde73: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:16:41.900040 26868 log.cc:1196] T dbacf6b7329f4252a8cf0deadfd18240 P b9b4e2cafa624126b2b8f0f05cedde73: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/wal/wals/dbacf6b7329f4252a8cf0deadfd18240
I20260811 05:16:41.900328 26868 ts_tablet_manager.cc:1939] T dbacf6b7329f4252a8cf0deadfd18240 P b9b4e2cafa624126b2b8f0f05cedde73: Deleting consensus metadata
I20260811 05:16:41.900588 26835 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260811 05:16:41.900688 26835 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260811 05:16:41.900730 26835 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:16:41.900749 26835 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:41.907275 26835 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.130:40211
I20260811 05:16:41.907389 27063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.130:40211 every 8 connection(s)
I20260811 05:16:41.907708 26835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
I20260811 05:16:41.908877 27069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:41.908425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.130:43689)
0811 05:16:41.908603 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:41.908713 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":76,"threads_started":1}
W20260811 05:16:41.909075 27064 heartbeater.cc:644] Failed to heartbeat to 127.28.143.190:36473 (0 consecutive failures): Network error: Failed to ping master at 127.28.143.190:36473: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
I20260811 05:16:41.909355 27069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:41.909269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.130:47057)
0811 05:16:41.909311 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:41.909338 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260811 05:16:41.909591 27069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:41.909511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.130:55889)
0811 05:16:41.909542 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:41.909561 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260811 05:16:41.909726 27064 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 05:16:41.910569 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 26835
I20260811 05:16:41.910830 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.131:44175
--local_ip_for_outbound_sockets=127.28.143.131
--tserver_master_addrs=127.28.143.190:36473
--webserver_port=37981
--webserver_interface=127.28.143.131
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:42519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:16:42.000362 27071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:16:42.000650 27071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:16:42.000680 27071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:16:42.002025 27071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:16:42.002105 27071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:16:42.002135 27071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.131
I20260811 05:16:42.003777 27071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:42519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.131:44175
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.143.131
--webserver_port=37981
--tserver_master_addrs=127.28.143.190:36473
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.131
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:16:42.004031 27071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:16:42.004253 27071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:16:42.009884 27082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:42.009889 27078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:42.009931 27080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:42.010249 27071 server_base.cc:1034] running on GCE node
I20260811 05:16:42.010493 27071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:16:42.010743 27071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:16:42.011920 27071 hybrid_clock.cc:648] HybridClock initialized: now 1786425402011905 us; error 31 us; skew 500 ppm
Aug 11 05:16:42 dist-test-slave-f7th krb5kdc[24572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425402, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:16:42.014770 27071 init.cc:377] Logged in from keytab as kudu/127.28.143.131@KRBTEST.COM (short username kudu)
I20260811 05:16:42.015313 27071 webserver.cc:460] Webserver started at http://127.28.143.131:37981/ using document root <none> and password file <none>
I20260811 05:16:42.015542 27071 fs_manager.cc:359] Metadata directory not provided
I20260811 05:16:42.015595 27071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:16:42.017220 27071 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 05:16:42.017918 27092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:42.018127 27071 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:16:42.018219 27071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/wal
uuid: "169da5b1aeea4a0a88c8ccab57cc2f14"
format_stamp: "Formatted at 2026-08-11 05:16:00 on dist-test-slave-f7th"
server_key: "cb97f7628eb515ea0b7ef627d81e2d18"
server_key_iv: "c5767a466c0e581804665a71927a559c"
server_key_version: "encryptionkey@0"
I20260811 05:16:42.018594 27071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:16:42.031333 27071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:16:42.033766 27071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:16:42.033898 27071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:16:42.034176 27071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:16:42.034509 27071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:16:42.034548 27071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:42.034579 27071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:16:42.034605 27071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:42.040961 27071 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.131:44175
I20260811 05:16:42.041043 27289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.131:44175 every 8 connection(s)
I20260811 05:16:42.041427 27071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
I20260811 05:16:42.042558 27292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:42.042156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.131:37101)
0811 05:16:42.042326 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:42.042405 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":82,"threads_started":1}
W20260811 05:16:42.043042 27290 heartbeater.cc:644] Failed to heartbeat to 127.28.143.190:36473 (0 consecutive failures): Network error: Failed to ping master at 127.28.143.190:36473: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
I20260811 05:16:42.043365 27292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:42.043249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.131:50059)
0811 05:16:42.043309 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:42.043342 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 05:16:42.043697 27292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:42.043570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.131:32849)
0811 05:16:42.043642 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:42.043674 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 05:16:42.043826 27290 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 05:16:42.047824 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 27071
I20260811 05:16:42.807402 27297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:42.807114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.129:42225)
0811 05:16:42.807269 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:42.807317 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":64,"threads_started":1}
I20260811 05:16:42.889192 27299 raft_consensus.cc:482] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:16:42.889364 27299 raft_consensus.cc:504] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9b4e2cafa624126b2b8f0f05cedde73" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 40211 } } peers { permanent_uuid: "169da5b1aeea4a0a88c8ccab57cc2f14" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44175 } } peers { permanent_uuid: "95bb8596089e4e69ba1427ac129f4a0f" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43851 } }
I20260811 05:16:42.889720 27299 leader_election.cc:288] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b9b4e2cafa624126b2b8f0f05cedde73 (127.28.143.130:40211), 169da5b1aeea4a0a88c8ccab57cc2f14 (127.28.143.131:44175)
Aug 11 05:16:42 dist-test-slave-f7th krb5kdc[24572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425401, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for kudu/127.28.143.130@KRBTEST.COM
Aug 11 05:16:42 dist-test-slave-f7th krb5kdc[24572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425401, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for kudu/127.28.143.131@KRBTEST.COM
I20260811 05:16:42.900758 27306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:42.890119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:58487 (local address 127.28.143.131:44175)
0811 05:16:42.890278 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:42.890288 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:42.890309 (+ 21us) server_negotiation.cc:407] Connection header received
0811 05:16:42.890708 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:42.890710 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:42.890766 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:42.890956 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:42.891717 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:42.892690 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:42.893739 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:42.894160 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:42.894263 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:42.897154 (+ 2891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:42.897159 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:42.897159 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:42.897172 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:42.898882 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:42.899321 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:42.899325 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:42.899326 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:42.899456 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:42.899688 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:42.899689 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:42.899689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:42.900108 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:42.900206 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:42.900488 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 05:16:42.900656 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":62,"threads_started":1}
I20260811 05:16:42.900780 27303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:42.890027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:52151 (local address 127.28.143.130:40211)
0811 05:16:42.890193 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:42.890206 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:42.890232 (+ 26us) server_negotiation.cc:407] Connection header received
0811 05:16:42.890732 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:42.890734 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:42.890786 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:42.890931 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:42.891713 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:42.892675 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:42.893743 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:42.894161 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:42.894263 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:42.896970 (+ 2707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:42.896975 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:42.896975 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:42.896990 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:42.898780 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:42.899322 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:42.899326 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:42.899326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:42.899437 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:42.899685 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:42.899686 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:42.899686 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:42.900073 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:42.900159 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:42.900457 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 05:16:42.900589 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":86,"threads_started":1}
I20260811 05:16:42.900789 27297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:42.889878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:40211 (local address 127.28.143.129:52151)
0811 05:16:42.889950 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:42.889963 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:42.890035 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:42.890975 (+ 940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:42.890978 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:42.891381 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:16:42.891588 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:42.891595 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:42.892778 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:42.892779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:42.893664 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:42.893667 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:42.894251 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:42.894252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:42.894380 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:42.895000 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:16:42.895009 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:16:42.896771 (+ 1762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:16:42.898881 (+ 2110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:42.898894 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:42.898897 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:42.899238 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:42.899506 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:42.899507 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:42.899507 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:42.899624 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:42.900145 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:16:42.900154 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:16:42.900293 (+ 139us) client_negotiation.cc:769] Sending connection context
0811 05:16:42.900399 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 05:16:42.900606 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":71}
I20260811 05:16:42.900843 27300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:42.889990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.131:44175 (local address 127.28.143.129:58487)
0811 05:16:42.890122 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:42.890131 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:42.890184 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:42.890976 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:42.890978 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:42.891382 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:16:42.891588 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:42.891595 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:42.892786 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:42.892787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:42.893642 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:42.893644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:42.894254 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:42.894255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:42.894380 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:42.895001 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:16:42.895009 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:16:42.897024 (+ 2015us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:16:42.898975 (+ 1951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:42.898976 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:42.898977 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:42.899237 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:42.899538 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:42.899538 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:42.899539 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:42.899624 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:42.900205 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:16:42.900207 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:16:42.900296 (+ 89us) client_negotiation.cc:769] Sending connection context
0811 05:16:42.900443 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 05:16:42.900703 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":158,"thread_start_us":62,"threads_started":1}
W20260811 05:16:42.901646 26627 leader_election.cc:341] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b9b4e2cafa624126b2b8f0f05cedde73 (127.28.143.130:40211): Not found: Tablet not found: dbacf6b7329f4252a8cf0deadfd18240
W20260811 05:16:42.902166 26627 leader_election.cc:341] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 169da5b1aeea4a0a88c8ccab57cc2f14 (127.28.143.131:44175): Not found: Tablet not found: dbacf6b7329f4252a8cf0deadfd18240
I20260811 05:16:42.902225 26627 leader_election.cc:302] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95bb8596089e4e69ba1427ac129f4a0f; no voters: 169da5b1aeea4a0a88c8ccab57cc2f14, b9b4e2cafa624126b2b8f0f05cedde73
I20260811 05:16:42.902338 27299 raft_consensus.cc:2726] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 05:16:42.910336 27307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:42.910098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.130:34265)
0811 05:16:42.910252 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:42.910296 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":92,"threads_started":1}
I20260811 05:16:43.044481 27310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:43.044194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.131:48279)
0811 05:16:43.044352 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:43.044435 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":68,"threads_started":1}
I20260811 05:16:43.808306 27311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:43.808004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.129:38875)
0811 05:16:43.808209 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:43.808256 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":69,"threads_started":1}
I20260811 05:16:43.911079 27317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:43.910830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.130:42107)
0811 05:16:43.910974 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:43.911030 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":77,"threads_started":1}
I20260811 05:16:44.045364 27318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:44.045090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.131:53371)
0811 05:16:44.045260 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:44.045314 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":70,"threads_started":1}
I20260811 05:16:44.809155 27320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:44.808899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.129:55013)
0811 05:16:44.809054 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:44.809105 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":71,"threads_started":1}
I20260811 05:16:44.911882 27323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:44.911623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.130:40261)
0811 05:16:44.911786 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:44.911835 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":69,"threads_started":1}
I20260811 05:16:45.005266 27329 raft_consensus.cc:482] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:16:45.005375 27329 raft_consensus.cc:504] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9b4e2cafa624126b2b8f0f05cedde73" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 40211 } } peers { permanent_uuid: "169da5b1aeea4a0a88c8ccab57cc2f14" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44175 } } peers { permanent_uuid: "95bb8596089e4e69ba1427ac129f4a0f" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43851 } }
I20260811 05:16:45.005555 27329 leader_election.cc:288] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b9b4e2cafa624126b2b8f0f05cedde73 (127.28.143.130:40211), 169da5b1aeea4a0a88c8ccab57cc2f14 (127.28.143.131:44175)
W20260811 05:16:45.005995 26627 leader_election.cc:341] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b9b4e2cafa624126b2b8f0f05cedde73 (127.28.143.130:40211): Not found: Tablet not found: dbacf6b7329f4252a8cf0deadfd18240
W20260811 05:16:45.006069 26627 leader_election.cc:341] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 169da5b1aeea4a0a88c8ccab57cc2f14 (127.28.143.131:44175): Not found: Tablet not found: dbacf6b7329f4252a8cf0deadfd18240
I20260811 05:16:45.006085 26627 leader_election.cc:302] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95bb8596089e4e69ba1427ac129f4a0f; no voters: 169da5b1aeea4a0a88c8ccab57cc2f14, b9b4e2cafa624126b2b8f0f05cedde73
I20260811 05:16:45.006153 27329 raft_consensus.cc:2726] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 05:16:45.046198 27330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:45.045920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.131:46317)
0811 05:16:45.046100 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:45.046148 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":103,"threads_started":1}
I20260811 05:16:45.809916 27334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:45.809648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.129:40151)
0811 05:16:45.809787 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:45.809866 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":63,"threads_started":1}
I20260811 05:16:45.912853 27336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:45.912536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.130:47609)
0811 05:16:45.912735 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:45.912799 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":128,"threads_started":1}
I20260811 05:16:46.046950 27338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:46.046723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.131:33881)
0811 05:16:46.046857 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:46.046903 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":70,"threads_started":1}
I20260811 05:16:46.810702 27339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:46.810457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.129:50629)
0811 05:16:46.810608 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:46.810652 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":69,"threads_started":1}
I20260811 05:16:46.820176 27340 raft_consensus.cc:482] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:16:46.820281 27340 raft_consensus.cc:504] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9b4e2cafa624126b2b8f0f05cedde73" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 40211 } } peers { permanent_uuid: "169da5b1aeea4a0a88c8ccab57cc2f14" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44175 } } peers { permanent_uuid: "95bb8596089e4e69ba1427ac129f4a0f" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43851 } }
I20260811 05:16:46.820418 27340 leader_election.cc:288] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b9b4e2cafa624126b2b8f0f05cedde73 (127.28.143.130:40211), 169da5b1aeea4a0a88c8ccab57cc2f14 (127.28.143.131:44175)
W20260811 05:16:46.820869 26627 leader_election.cc:341] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b9b4e2cafa624126b2b8f0f05cedde73 (127.28.143.130:40211): Not found: Tablet not found: dbacf6b7329f4252a8cf0deadfd18240
W20260811 05:16:46.820935 26627 leader_election.cc:341] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 169da5b1aeea4a0a88c8ccab57cc2f14 (127.28.143.131:44175): Not found: Tablet not found: dbacf6b7329f4252a8cf0deadfd18240
I20260811 05:16:46.820995 26627 leader_election.cc:302] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95bb8596089e4e69ba1427ac129f4a0f; no voters: 169da5b1aeea4a0a88c8ccab57cc2f14, b9b4e2cafa624126b2b8f0f05cedde73
I20260811 05:16:46.821074 27340 raft_consensus.cc:2726] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 05:16:46.913757 27341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:46.913508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.130:56049)
0811 05:16:46.913652 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:46.913708 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":72,"threads_started":1}
I20260811 05:16:47.047778 27344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:47.047546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.131:37825)
0811 05:16:47.047682 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:47.047728 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":67,"threads_started":1}
I20260811 05:16:47.811489 27345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:47.811242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.129:50739)
0811 05:16:47.811400 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:47.811441 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":73,"threads_started":1}
I20260811 05:16:47.914604 27346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:47.914333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.130:35635)
0811 05:16:47.914500 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:47.914553 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":73,"threads_started":1}
I20260811 05:16:48.048707 27347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:48.048425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.131:44873)
0811 05:16:48.048572 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:48.048650 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":70,"threads_started":1}
I20260811 05:16:48.812242 27350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:48.812016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.129:41891)
0811 05:16:48.812155 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:48.812193 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":68,"threads_started":1}
I20260811 05:16:48.915530 27353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:48.915247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.130:47393)
0811 05:16:48.915430 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:48.915479 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":90,"threads_started":1}
I20260811 05:16:49.049487 27356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:49.049190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.131:59375)
0811 05:16:49.049378 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:49.049428 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":88,"threads_started":1}
I20260811 05:16:49.813133 27358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:49.812823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.129:36159)
0811 05:16:49.813035 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:49.813084 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":74,"threads_started":1}
I20260811 05:16:49.856316 27359 raft_consensus.cc:482] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:16:49.856446 27359 raft_consensus.cc:504] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9b4e2cafa624126b2b8f0f05cedde73" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 40211 } } peers { permanent_uuid: "169da5b1aeea4a0a88c8ccab57cc2f14" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44175 } } peers { permanent_uuid: "95bb8596089e4e69ba1427ac129f4a0f" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43851 } }
I20260811 05:16:49.856648 27359 leader_election.cc:288] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b9b4e2cafa624126b2b8f0f05cedde73 (127.28.143.130:40211), 169da5b1aeea4a0a88c8ccab57cc2f14 (127.28.143.131:44175)
W20260811 05:16:49.857131 26627 leader_election.cc:341] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b9b4e2cafa624126b2b8f0f05cedde73 (127.28.143.130:40211): Not found: Tablet not found: dbacf6b7329f4252a8cf0deadfd18240
W20260811 05:16:49.857208 26627 leader_election.cc:341] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 169da5b1aeea4a0a88c8ccab57cc2f14 (127.28.143.131:44175): Not found: Tablet not found: dbacf6b7329f4252a8cf0deadfd18240
I20260811 05:16:49.857221 26627 leader_election.cc:302] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95bb8596089e4e69ba1427ac129f4a0f; no voters: 169da5b1aeea4a0a88c8ccab57cc2f14, b9b4e2cafa624126b2b8f0f05cedde73
I20260811 05:16:49.857317 27359 raft_consensus.cc:2726] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 05:16:49.916400 27362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:49.916131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.130:55887)
0811 05:16:49.916297 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:49.916349 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":66,"threads_started":1}
I20260811 05:16:50.050305 27363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:50.050051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.131:42331)
0811 05:16:50.050188 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:50.050235 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":68,"threads_started":1}
I20260811 05:16:50.813966 27364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:50.813711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.129:33891)
0811 05:16:50.813867 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:50.813916 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":73,"threads_started":1}
I20260811 05:16:50.917265 27371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:50.917002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.130:57623)
0811 05:16:50.917169 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:50.917216 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":93,"threads_started":1}
I20260811 05:16:51.051225 27372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:51.050970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.131:37113)
0811 05:16:51.051120 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:51.051171 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":71,"threads_started":1}
I20260811 05:16:51.643002 27373 raft_consensus.cc:482] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:16:51.643114 27373 raft_consensus.cc:504] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9b4e2cafa624126b2b8f0f05cedde73" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 40211 } } peers { permanent_uuid: "169da5b1aeea4a0a88c8ccab57cc2f14" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44175 } } peers { permanent_uuid: "95bb8596089e4e69ba1427ac129f4a0f" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43851 } }
I20260811 05:16:51.643282 27373 leader_election.cc:288] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b9b4e2cafa624126b2b8f0f05cedde73 (127.28.143.130:40211), 169da5b1aeea4a0a88c8ccab57cc2f14 (127.28.143.131:44175)
W20260811 05:16:51.643705 26627 leader_election.cc:341] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 169da5b1aeea4a0a88c8ccab57cc2f14 (127.28.143.131:44175): Not found: Tablet not found: dbacf6b7329f4252a8cf0deadfd18240
W20260811 05:16:51.643777 26627 leader_election.cc:341] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b9b4e2cafa624126b2b8f0f05cedde73 (127.28.143.130:40211): Not found: Tablet not found: dbacf6b7329f4252a8cf0deadfd18240
I20260811 05:16:51.643795 26627 leader_election.cc:302] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95bb8596089e4e69ba1427ac129f4a0f; no voters: 169da5b1aeea4a0a88c8ccab57cc2f14, b9b4e2cafa624126b2b8f0f05cedde73
I20260811 05:16:51.643885 27373 raft_consensus.cc:2726] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 05:16:51.814884 27374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:51.814610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.129:54641)
0811 05:16:51.814783 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:51.814836 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":74,"threads_started":1}
I20260811 05:16:51.917948 27378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:51.917702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.130:33989)
0811 05:16:51.917849 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:51.917895 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":75,"threads_started":1}
I20260811 05:16:52.052156 27379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:52.051852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.131:49727)
0811 05:16:52.052053 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:52.052106 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":109,"threads_started":1}
I20260811 05:16:52.815793 27380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:52.815463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.129:41963)
0811 05:16:52.815663 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:52.815714 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":93,"threads_started":1}
I20260811 05:16:52.918839 27381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:52.918551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.130:50535)
0811 05:16:52.918705 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:52.918749 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":69,"threads_started":1}
I20260811 05:16:53.053140 27385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:53.052837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.131:53349)
0811 05:16:53.053038 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:53.053090 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":117,"threads_started":1}
I20260811 05:16:53.816787 27386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:53.816474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.129:53735)
0811 05:16:53.816689 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:53.816738 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":77,"threads_started":1}
I20260811 05:16:53.830564 27388 raft_consensus.cc:482] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:16:53.830693 27388 raft_consensus.cc:504] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9b4e2cafa624126b2b8f0f05cedde73" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 40211 } } peers { permanent_uuid: "169da5b1aeea4a0a88c8ccab57cc2f14" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44175 } } peers { permanent_uuid: "95bb8596089e4e69ba1427ac129f4a0f" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43851 } }
I20260811 05:16:53.830860 27388 leader_election.cc:288] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b9b4e2cafa624126b2b8f0f05cedde73 (127.28.143.130:40211), 169da5b1aeea4a0a88c8ccab57cc2f14 (127.28.143.131:44175)
W20260811 05:16:53.831328 26627 leader_election.cc:341] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b9b4e2cafa624126b2b8f0f05cedde73 (127.28.143.130:40211): Not found: Tablet not found: dbacf6b7329f4252a8cf0deadfd18240
W20260811 05:16:53.831409 26627 leader_election.cc:341] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 169da5b1aeea4a0a88c8ccab57cc2f14 (127.28.143.131:44175): Not found: Tablet not found: dbacf6b7329f4252a8cf0deadfd18240
I20260811 05:16:53.831425 26627 leader_election.cc:302] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95bb8596089e4e69ba1427ac129f4a0f; no voters: 169da5b1aeea4a0a88c8ccab57cc2f14, b9b4e2cafa624126b2b8f0f05cedde73
I20260811 05:16:53.831529 27388 raft_consensus.cc:2726] T dbacf6b7329f4252a8cf0deadfd18240 P 95bb8596089e4e69ba1427ac129f4a0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 05:16:53.919582 27390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:53.919334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.130:51899)
0811 05:16:53.919500 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:53.919534 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":79,"threads_started":1}
I20260811 05:16:54.054047 27391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:54.053737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.131:55545)
0811 05:16:54.053913 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:54.053965 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":77,"threads_started":1}
I20260811 05:16:54.817544 27392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:54.817317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.129:36283)
0811 05:16:54.817446 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:54.817494 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":69,"threads_started":1}
I20260811 05:16:54.920338 27393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:54.920081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.130:54215)
0811 05:16:54.920241 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:54.920286 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":67,"threads_started":1}
I20260811 05:16:55.054939 27396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:55.054667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.131:41583)
0811 05:16:55.054840 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:55.054890 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":86,"threads_started":1}
I20260811 05:16:55.818341 27397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:55.818117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.129:44685)
0811 05:16:55.818247 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:55.818294 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":72,"threads_started":1}
I20260811 05:16:55.921123 27398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:55.920857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.130:36131)
0811 05:16:55.921012 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:55.921066 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":70,"threads_started":1}
I20260811 05:16:56.055787 27399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:56.055497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.131:53539)
0811 05:16:56.055677 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:56.055730 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":94,"threads_started":1}
I20260811 05:16:56.819099 27400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:56.818840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.129:54401)
0811 05:16:56.818988 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:56.819038 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":57,"threads_started":1}
I20260811 05:16:56.921916 27401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:56.921658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.130:46897)
0811 05:16:56.921812 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:56.921850 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":68,"threads_started":1}
I20260811 05:16:57.049237 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 26534
I20260811 05:16:57.062209 27402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:57.060023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:36473 (local address 127.28.143.131:50405)
0811 05:16:57.062088 (+ 2065us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:57.062147 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.190:36473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2039,"thread_start_us":1981,"threads_started":1}
I20260811 05:16:57.066757 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 26835
I20260811 05:16:57.079283 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 27071
2026-08-11T05:16:57Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58923 ms)
[----------] 30 tests from SecurityITest (258035 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:16:57 dist-test-slave-f7th krb5kdc[27409](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:16:57 dist-test-slave-f7th krb5kdc[27409](info): set up 2 sockets
Aug 11 05:16:57 dist-test-slave-f7th krb5kdc[27409](info): commencing operation
krb5kdc: starting...
W20260811 05:16:58.317725 29246 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Aug 11 05:16:58 dist-test-slave-f7th krb5kdc[27409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425418, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:16:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:16:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
I20260811 05:16:58.453190 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:44905
--webserver_interface=127.28.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:36087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:44905
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:16:58.538072 27429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:16:58.538316 27429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:16:58.538347 27429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:16:58.539510 27429 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:16:58.539558 27429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:16:58.539577 27429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:16:58.539593 27429 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:16:58.539608 27429 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:16:58.541162 27429 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:36087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:44905
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:44905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:16:58.541428 27429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:16:58.541643 27429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:16:58.546553 27446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:58.546639 27439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:58.546553 27443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:58.546742 27429 server_base.cc:1034] running on GCE node
I20260811 05:16:58.547084 27429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:16:58.547400 27429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:16:58.548563 27429 hybrid_clock.cc:648] HybridClock initialized: now 1786425418548539 us; error 33 us; skew 500 ppm
Aug 11 05:16:58 dist-test-slave-f7th krb5kdc[27409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425418, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:16:58.551216 27429 init.cc:377] Logged in from keytab as kudu/127.28.143.190@KRBTEST.COM (short username kudu)
I20260811 05:16:58.551703 27429 webserver.cc:460] Webserver started at http://127.28.143.190:34069/ using document root <none> and password file <none>
I20260811 05:16:58.551928 27429 fs_manager.cc:359] Metadata directory not provided
I20260811 05:16:58.551980 27429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:16:58.552115 27429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:16:58.553243 27429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "9114e391bc2c4e8d8444d94da2367c4e"
format_stamp: "Formatted at 2026-08-11 05:16:58 on dist-test-slave-f7th"
server_key: "17113fbb40b70464d34f601a704e6c1c"
server_key_iv: "52078b39d3b7387764fa5e395ab57a0f"
server_key_version: "encryptionkey@0"
I20260811 05:16:58.553586 27429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "9114e391bc2c4e8d8444d94da2367c4e"
format_stamp: "Formatted at 2026-08-11 05:16:58 on dist-test-slave-f7th"
server_key: "17113fbb40b70464d34f601a704e6c1c"
server_key_iv: "52078b39d3b7387764fa5e395ab57a0f"
server_key_version: "encryptionkey@0"
I20260811 05:16:58.554955 27429 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 05:16:58.555716 27455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:58.555917 27429 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:16:58.556020 27429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "9114e391bc2c4e8d8444d94da2367c4e"
format_stamp: "Formatted at 2026-08-11 05:16:58 on dist-test-slave-f7th"
server_key: "17113fbb40b70464d34f601a704e6c1c"
server_key_iv: "52078b39d3b7387764fa5e395ab57a0f"
server_key_version: "encryptionkey@0"
I20260811 05:16:58.556088 27429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:16:58.564859 27429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:16:58.567072 27429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:16:58.567233 27429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:16:58.571496 27429 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:44905
I20260811 05:16:58.571539 27553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:44905 every 8 connection(s)
I20260811 05:16:58.571907 27429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:16:58.572860 27559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:16:58.575227 27559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9114e391bc2c4e8d8444d94da2367c4e: Bootstrap starting.
I20260811 05:16:58.575784 27559 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9114e391bc2c4e8d8444d94da2367c4e: Neither blocks nor log segments found. Creating new log.
I20260811 05:16:58.576042 27559 log.cc:824] T 00000000000000000000000000000000 P 9114e391bc2c4e8d8444d94da2367c4e: Log is configured to *not* fsync() on all Append() calls
I20260811 05:16:58.576783 27559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9114e391bc2c4e8d8444d94da2367c4e: No bootstrap required, opened a new log
I20260811 05:16:58.578359 27559 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9114e391bc2c4e8d8444d94da2367c4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9114e391bc2c4e8d8444d94da2367c4e" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 44905 } }
I20260811 05:16:58.578502 27559 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9114e391bc2c4e8d8444d94da2367c4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:16:58.578528 27559 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9114e391bc2c4e8d8444d94da2367c4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9114e391bc2c4e8d8444d94da2367c4e, State: Initialized, Role: FOLLOWER
I20260811 05:16:58.578634 27559 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9114e391bc2c4e8d8444d94da2367c4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9114e391bc2c4e8d8444d94da2367c4e" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 44905 } }
I20260811 05:16:58.578687 27559 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9114e391bc2c4e8d8444d94da2367c4e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:16:58.578707 27559 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9114e391bc2c4e8d8444d94da2367c4e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:16:58.578740 27559 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9114e391bc2c4e8d8444d94da2367c4e [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:16:58.579243 27559 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9114e391bc2c4e8d8444d94da2367c4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9114e391bc2c4e8d8444d94da2367c4e" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 44905 } }
I20260811 05:16:58.579334 27559 leader_election.cc:302] T 00000000000000000000000000000000 P 9114e391bc2c4e8d8444d94da2367c4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9114e391bc2c4e8d8444d94da2367c4e; no voters:
I20260811 05:16:58.579475 27559 leader_election.cc:288] T 00000000000000000000000000000000 P 9114e391bc2c4e8d8444d94da2367c4e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:16:58.579538 27565 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9114e391bc2c4e8d8444d94da2367c4e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:16:58.579704 27565 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9114e391bc2c4e8d8444d94da2367c4e [term 1 LEADER]: Becoming Leader. State: Replica: 9114e391bc2c4e8d8444d94da2367c4e, State: Running, Role: LEADER
I20260811 05:16:58.579771 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 27429
I20260811 05:16:58.579787 27559 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9114e391bc2c4e8d8444d94da2367c4e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:16:58.579823 27565 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9114e391bc2c4e8d8444d94da2367c4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9114e391bc2c4e8d8444d94da2367c4e" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 44905 } }
I20260811 05:16:58.579881 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:16:58.580073 29246 external_mini_cluster.cc:1428] Setting key 3d3b15916a9d2e4ef9654a305a644636
I20260811 05:16:58.580145 27565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9114e391bc2c4e8d8444d94da2367c4e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9114e391bc2c4e8d8444d94da2367c4e. Latest consensus state: current_term: 1 leader_uuid: "9114e391bc2c4e8d8444d94da2367c4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9114e391bc2c4e8d8444d94da2367c4e" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 44905 } } }
I20260811 05:16:58.580214 27565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9114e391bc2c4e8d8444d94da2367c4e [sys.catalog]: This master's current role is: LEADER
I20260811 05:16:58.580523 27566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9114e391bc2c4e8d8444d94da2367c4e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9114e391bc2c4e8d8444d94da2367c4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9114e391bc2c4e8d8444d94da2367c4e" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 44905 } } }
I20260811 05:16:58.580583 27566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9114e391bc2c4e8d8444d94da2367c4e [sys.catalog]: This master's current role is: LEADER
W20260811 05:16:58.581887 27598 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9114e391bc2c4e8d8444d94da2367c4e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:16:58.581938 27598 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:16:58.581979 27575 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:16:58.582129 27575 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:16:58.583729 27575 catalog_manager.cc:1347] Generated new cluster ID: 4d0b520c548049058de60c4789ac1286
I20260811 05:16:58.583787 27575 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 05:16:58 dist-test-slave-f7th krb5kdc[27409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425418, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:16:58.592289 27588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:58.580768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34568 (local address 127.28.143.190:44905)
0811 05:16:58.581277 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:58.581288 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:58.581331 (+ 43us) server_negotiation.cc:407] Connection header received
0811 05:16:58.581523 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:58.581531 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:58.581605 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:58.581764 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:58.582535 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:58.583663 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:58.584798 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:58.585468 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:58.585577 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:58.588542 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:58.588548 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:58.588548 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:58.588564 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:58.590603 (+ 2039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:58.590995 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:58.590999 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:58.590999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:58.591137 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:58.591313 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:58.591314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:58.591314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:58.591661 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:58.591832 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:58.591903 (+ 71us) server_negotiation.cc:299] Negotiation successful
0811 05:16:58.592054 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":77,"threads_started":1}
I20260811 05:16:58.601868 27575 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:16:58.603458 27575 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:16:58.620080 27575 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9114e391bc2c4e8d8444d94da2367c4e: Generated new TSK 0
I20260811 05:16:58.620306 27575 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.143.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab.
Entry for principal kudu/127.28.143.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab.
I20260811 05:16:58.655401 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.129:0
--local_ip_for_outbound_sockets=127.28.143.129
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:44905
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:36087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:16:58.762274 27607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:16:58.762506 27607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:16:58.762533 27607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:16:58.763806 27607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:16:58.763859 27607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:16:58.763888 27607 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.129
I20260811 05:16:58.765563 27607 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:36087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:44905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.129
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:16:58.765839 27607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:16:58.766096 27607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:16:58.771509 27620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:58.771595 27607 server_base.cc:1034] running on GCE node
W20260811 05:16:58.771524 27616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:58.771520 27618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:58.771910 27607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:16:58.772234 27607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:16:58.773380 27607 hybrid_clock.cc:648] HybridClock initialized: now 1786425418773366 us; error 30 us; skew 500 ppm
Aug 11 05:16:58 dist-test-slave-f7th krb5kdc[27409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425418, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:16:58.776218 27607 init.cc:377] Logged in from keytab as kudu/127.28.143.129@KRBTEST.COM (short username kudu)
I20260811 05:16:58.776729 27607 webserver.cc:460] Webserver started at http://127.28.143.129:33123/ using document root <none> and password file <none>
I20260811 05:16:58.776963 27607 fs_manager.cc:359] Metadata directory not provided
I20260811 05:16:58.777005 27607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:16:58.777109 27607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:16:58.778172 27607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-0/data/instance:
uuid: "efe92987c46840cfbc53d2114b8533f5"
format_stamp: "Formatted at 2026-08-11 05:16:58 on dist-test-slave-f7th"
server_key: "84dcb5fc91ce4206341c4ae0bf669be3"
server_key_iv: "3687646c979b0d8bff0532ea08149c3b"
server_key_version: "encryptionkey@0"
I20260811 05:16:58.778483 27607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance:
uuid: "efe92987c46840cfbc53d2114b8533f5"
format_stamp: "Formatted at 2026-08-11 05:16:58 on dist-test-slave-f7th"
server_key: "84dcb5fc91ce4206341c4ae0bf669be3"
server_key_iv: "3687646c979b0d8bff0532ea08149c3b"
server_key_version: "encryptionkey@0"
I20260811 05:16:58.779806 27607 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 05:16:58.780663 27636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:58.780908 27607 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:16:58.781044 27607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-0/wal
uuid: "efe92987c46840cfbc53d2114b8533f5"
format_stamp: "Formatted at 2026-08-11 05:16:58 on dist-test-slave-f7th"
server_key: "84dcb5fc91ce4206341c4ae0bf669be3"
server_key_iv: "3687646c979b0d8bff0532ea08149c3b"
server_key_version: "encryptionkey@0"
I20260811 05:16:58.781139 27607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:16:58.803231 27607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:16:58.805553 27607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:16:58.805687 27607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:16:58.805955 27607 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:16:58.806290 27607 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:16:58.806326 27607 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:58.806352 27607 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:16:58.806375 27607 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:58.812328 27607 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.129:42875
I20260811 05:16:58.812388 27831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.129:42875 every 8 connection(s)
I20260811 05:16:58.812736 27607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
I20260811 05:16:58.812937 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 27607
I20260811 05:16:58.813047 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance
I20260811 05:16:58.813251 29246 external_mini_cluster.cc:1428] Setting key aef69fd6bbe4682c1e3660ca954cb1c9
Aug 11 05:16:58 dist-test-slave-f7th krb5kdc[27409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425418, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.28.143.130@KRBTEST.COM; defaulting to no policy
I20260811 05:16:58.822841 27588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:58.813571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:48541 (local address 127.28.143.190:44905)
0811 05:16:58.813620 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:58.813623 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:58.813839 (+ 216us) server_negotiation.cc:407] Connection header received
0811 05:16:58.814476 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:58.814477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:58.814488 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:58.814537 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:58.815752 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:58.816259 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:58.817650 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:58.817855 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:58.817918 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:58.820115 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:58.820117 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:58.820117 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:58.820125 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:58.821491 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:58.821889 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:58.821890 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:58.821890 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:58.821952 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:58.822187 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:58.822188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:58.822188 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:58.822305 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:58.822363 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:58.822649 (+ 286us) server_negotiation.cc:299] Negotiation successful
0811 05:16:58.822747 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 05:16:58.823019 27836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:58.813562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:44905 (local address 127.28.143.129:48541)
0811 05:16:58.813721 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:58.813761 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:58.814358 (+ 597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:58.814609 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:58.814613 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:58.815373 (+ 760us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:16:58.815653 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:58.815660 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:58.816352 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:58.816354 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:58.817560 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:58.817563 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:58.817955 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:58.817956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:58.818082 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:58.818598 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:16:58.818608 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:16:58.820023 (+ 1415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:16:58.821571 (+ 1548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:58.821579 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:58.821582 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:58.821817 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:58.822015 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:58.822016 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:58.822016 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:58.822130 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:58.822362 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:16:58.822370 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:16:58.822522 (+ 152us) client_negotiation.cc:769] Sending connection context
0811 05:16:58.822615 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 05:16:58.822782 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":78,"threads_started":1}
I20260811 05:16:58.823495 27833 heartbeater.cc:347] Connected to a master server at 127.28.143.190:44905
I20260811 05:16:58.823597 27833 heartbeater.cc:464] Registering TS with master...
I20260811 05:16:58.823789 27833 heartbeater.cc:511] Master 127.28.143.190:44905 requested a full tablet report, sending...
I20260811 05:16:58.824366 27489 ts_manager.cc:194] Registered new tserver with Master: efe92987c46840cfbc53d2114b8533f5 (127.28.143.129:42875)
I20260811 05:16:58.825065 27489 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.129@KRBTEST.COM'} at 127.28.143.129:48541
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab.
Entry for principal kudu/127.28.143.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab.
I20260811 05:16:58.843838 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.130:0
--local_ip_for_outbound_sockets=127.28.143.130
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:44905
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:36087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:16:58.929394 27842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:16:58.929632 27842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:16:58.929657 27842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:16:58.930927 27842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:16:58.930991 27842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:16:58.931020 27842 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.130
I20260811 05:16:58.932583 27842 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:36087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:44905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.130
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:16:58.932844 27842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:16:58.933048 27842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:16:58.938247 27858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:58.938300 27856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:58.938247 27852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:58.938418 27842 server_base.cc:1034] running on GCE node
I20260811 05:16:58.938707 27842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:16:58.939019 27842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:16:58.940187 27842 hybrid_clock.cc:648] HybridClock initialized: now 1786425418940161 us; error 38 us; skew 500 ppm
Aug 11 05:16:58 dist-test-slave-f7th krb5kdc[27409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425418, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:16:58.942934 27842 init.cc:377] Logged in from keytab as kudu/127.28.143.130@KRBTEST.COM (short username kudu)
I20260811 05:16:58.943400 27842 webserver.cc:460] Webserver started at http://127.28.143.130:42395/ using document root <none> and password file <none>
I20260811 05:16:58.943614 27842 fs_manager.cc:359] Metadata directory not provided
I20260811 05:16:58.943665 27842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:16:58.943795 27842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:16:58.944983 27842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-1/data/instance:
uuid: "816585fd4c594406b66bf2603953639b"
format_stamp: "Formatted at 2026-08-11 05:16:58 on dist-test-slave-f7th"
server_key: "7a69fb28db0351de83b168b082ced6d9"
server_key_iv: "08c5bf2c99d4ed37a700298733703628"
server_key_version: "encryptionkey@0"
I20260811 05:16:58.945418 27842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance:
uuid: "816585fd4c594406b66bf2603953639b"
format_stamp: "Formatted at 2026-08-11 05:16:58 on dist-test-slave-f7th"
server_key: "7a69fb28db0351de83b168b082ced6d9"
server_key_iv: "08c5bf2c99d4ed37a700298733703628"
server_key_version: "encryptionkey@0"
I20260811 05:16:58.946885 27842 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 05:16:58.947659 27872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:58.947870 27842 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:16:58.947973 27842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-1/wal
uuid: "816585fd4c594406b66bf2603953639b"
format_stamp: "Formatted at 2026-08-11 05:16:58 on dist-test-slave-f7th"
server_key: "7a69fb28db0351de83b168b082ced6d9"
server_key_iv: "08c5bf2c99d4ed37a700298733703628"
server_key_version: "encryptionkey@0"
I20260811 05:16:58.948038 27842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:16:58.960167 27842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:16:58.962409 27842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:16:58.962566 27842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:16:58.962805 27842 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:16:58.963126 27842 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:16:58.963163 27842 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:58.963193 27842 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:16:58.963218 27842 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:58.969187 27842 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.130:41305
I20260811 05:16:58.969247 28074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.130:41305 every 8 connection(s)
I20260811 05:16:58.969581 27842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
I20260811 05:16:58.970604 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 27842
I20260811 05:16:58.970705 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance
I20260811 05:16:58.970914 29246 external_mini_cluster.cc:1428] Setting key 5043d102f1297bf4a99b429aa8e4fcf3
Aug 11 05:16:58 dist-test-slave-f7th krb5kdc[27409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425418, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:16:58.980255 27588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:58.970462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:60993 (local address 127.28.143.190:44905)
0811 05:16:58.970577 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:58.970579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:58.970629 (+ 50us) server_negotiation.cc:407] Connection header received
0811 05:16:58.971238 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:58.971239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:58.971253 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:58.971302 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:58.972338 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:58.972912 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:58.974275 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:58.974495 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:58.974564 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:58.977248 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:58.977250 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:58.977250 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:58.977258 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:58.978639 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:58.979159 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:58.979159 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:58.979160 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:58.979223 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:58.979511 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:58.979512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:58.979512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:58.979638 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:58.979726 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:58.980057 (+ 331us) server_negotiation.cc:299] Negotiation successful
0811 05:16:58.980158 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
WARNING: no policy specified for kudu/127.28.143.131@KRBTEST.COM; defaulting to no policy
I20260811 05:16:58.980504 28078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:58.970344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:44905 (local address 127.28.143.130:60993)
0811 05:16:58.970512 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:58.970557 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:58.971113 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:58.971377 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:58.971381 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:58.972037 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:16:58.972242 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:58.972248 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:58.973000 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:58.973001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:58.974162 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:58.974166 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:58.974592 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:58.974594 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:58.974743 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:58.975312 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:16:58.975324 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:16:58.977116 (+ 1792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:16:58.978734 (+ 1618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:58.978743 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:58.978746 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:58.979086 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:58.979289 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:58.979289 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:58.979290 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:58.979436 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:58.979736 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:16:58.979745 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:16:58.979885 (+ 140us) client_negotiation.cc:769] Sending connection context
0811 05:16:58.980032 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 05:16:58.980207 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":69,"threads_started":1}
I20260811 05:16:58.981130 28075 heartbeater.cc:347] Connected to a master server at 127.28.143.190:44905
I20260811 05:16:58.981235 28075 heartbeater.cc:464] Registering TS with master...
I20260811 05:16:58.981438 28075 heartbeater.cc:511] Master 127.28.143.190:44905 requested a full tablet report, sending...
I20260811 05:16:58.981889 27489 ts_manager.cc:194] Registered new tserver with Master: 816585fd4c594406b66bf2603953639b (127.28.143.130:41305)
I20260811 05:16:58.982259 27489 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.130@KRBTEST.COM'} at 127.28.143.130:60993
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab.
Entry for principal kudu/127.28.143.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab.
I20260811 05:16:59.001672 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.131:0
--local_ip_for_outbound_sockets=127.28.143.131
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:44905
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:36087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:16:59.086568 28082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:16:59.086818 28082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:16:59.086861 28082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:16:59.088068 28082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:16:59.088111 28082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:16:59.088141 28082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.131
I20260811 05:16:59.089743 28082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:36087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:44905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.131
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:16:59.089962 28082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:16:59.090163 28082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:16:59.095444 28090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:59.095441 28091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:59.095441 28097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:59.095757 28082 server_base.cc:1034] running on GCE node
I20260811 05:16:59.095968 28082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:16:59.096227 28082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:16:59.097366 28082 hybrid_clock.cc:648] HybridClock initialized: now 1786425419097351 us; error 29 us; skew 500 ppm
Aug 11 05:16:59 dist-test-slave-f7th krb5kdc[27409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425419, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:16:59.100035 28082 init.cc:377] Logged in from keytab as kudu/127.28.143.131@KRBTEST.COM (short username kudu)
I20260811 05:16:59.100500 28082 webserver.cc:460] Webserver started at http://127.28.143.131:43281/ using document root <none> and password file <none>
I20260811 05:16:59.100775 28082 fs_manager.cc:359] Metadata directory not provided
I20260811 05:16:59.100828 28082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:16:59.100960 28082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:16:59.102068 28082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-2/data/instance:
uuid: "783a644a6c7c4088a5b6edd32aaed233"
format_stamp: "Formatted at 2026-08-11 05:16:59 on dist-test-slave-f7th"
server_key: "f1495e8e26343667dacae98c9bf50de2"
server_key_iv: "12bb5a5fb9ab7245dab6227e31713507"
server_key_version: "encryptionkey@0"
I20260811 05:16:59.102419 28082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance:
uuid: "783a644a6c7c4088a5b6edd32aaed233"
format_stamp: "Formatted at 2026-08-11 05:16:59 on dist-test-slave-f7th"
server_key: "f1495e8e26343667dacae98c9bf50de2"
server_key_iv: "12bb5a5fb9ab7245dab6227e31713507"
server_key_version: "encryptionkey@0"
I20260811 05:16:59.103796 28082 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 05:16:59.104749 28110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:59.104934 28082 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:16:59.105038 28082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-2/wal
uuid: "783a644a6c7c4088a5b6edd32aaed233"
format_stamp: "Formatted at 2026-08-11 05:16:59 on dist-test-slave-f7th"
server_key: "f1495e8e26343667dacae98c9bf50de2"
server_key_iv: "12bb5a5fb9ab7245dab6227e31713507"
server_key_version: "encryptionkey@0"
I20260811 05:16:59.105120 28082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:16:59.131067 28082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:16:59.133457 28082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:16:59.133637 28082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:16:59.133905 28082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:16:59.134246 28082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:16:59.134284 28082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:59.134315 28082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:16:59.134341 28082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:59.140933 28082 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.131:39545
I20260811 05:16:59.140981 28320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.131:39545 every 8 connection(s)
I20260811 05:16:59.141396 28082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
Aug 11 05:16:59 dist-test-slave-f7th krb5kdc[27409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425419, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:16:59.148106 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 28082
I20260811 05:16:59.148216 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance
I20260811 05:16:59.148422 29246 external_mini_cluster.cc:1428] Setting key db6374a40c1e1c4df0e0c3a6b1df27c8
I20260811 05:16:59.151742 27588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:59.142222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:58573 (local address 127.28.143.190:44905)
0811 05:16:59.142290 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:59.142292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:59.142502 (+ 210us) server_negotiation.cc:407] Connection header received
0811 05:16:59.143074 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:59.143076 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:59.143089 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:59.143150 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:59.144250 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.144847 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.145918 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.146132 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.146201 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:59.148733 (+ 2532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:59.148735 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:59.148735 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:59.148746 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:59.150329 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:59.150810 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:59.150810 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:59.150811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:59.150862 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:59.151175 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:59.151176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:59.151177 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:59.151295 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:59.151348 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:59.151608 (+ 260us) server_negotiation.cc:299] Negotiation successful
0811 05:16:59.151676 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 05:16:59.151907 28327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:59.142177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:44905 (local address 127.28.143.131:58573)
0811 05:16:59.142370 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:59.142418 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:59.142899 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:59.143265 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:59.143269 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:59.143917 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:16:59.144127 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:59.144132 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.144970 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.144971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:59.145811 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:59.145813 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.146219 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.146220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:59.146338 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:59.146904 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:16:59.146915 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:16:59.148602 (+ 1687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:16:59.150428 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:59.150435 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:59.150437 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:59.150746 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:59.150910 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:59.150910 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:59.150911 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:59.151114 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:59.151345 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:16:59.151348 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:16:59.151493 (+ 145us) client_negotiation.cc:769] Sending connection context
0811 05:16:59.151574 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 05:16:59.151708 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":91,"threads_started":1}
I20260811 05:16:59.152375 28321 heartbeater.cc:347] Connected to a master server at 127.28.143.190:44905
I20260811 05:16:59.152477 28321 heartbeater.cc:464] Registering TS with master...
I20260811 05:16:59.152706 28321 heartbeater.cc:511] Master 127.28.143.190:44905 requested a full tablet report, sending...
I20260811 05:16:59.153172 27489 ts_manager.cc:194] Registered new tserver with Master: 783a644a6c7c4088a5b6edd32aaed233 (127.28.143.131:39545)
I20260811 05:16:59.153570 27489 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.131@KRBTEST.COM'} at 127.28.143.131:58573
I20260811 05:16:59.160724 29246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:16:59.168699 27588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:59.162289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34582 (local address 127.28.143.190:44905)
0811 05:16:59.162400 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:59.162403 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:59.162455 (+ 52us) server_negotiation.cc:407] Connection header received
0811 05:16:59.162528 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:59.162529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:59.162540 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:59.162584 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:59.163186 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.163688 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.164465 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.164723 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.164778 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:59.166049 (+ 1271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:59.166051 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:59.166051 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:59.166059 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:59.167480 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:59.167855 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:59.167856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:59.167857 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:59.167938 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:59.168113 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:59.168114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:59.168114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:59.168283 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:59.168375 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:59.168494 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 05:16:59.168576 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 05:16:59.170169 27489 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34582:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:16:59.171768 27489 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:16:59.178283 28341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:59.174105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:42875 (local address 127.0.0.1:42260)
0811 05:16:59.174489 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:59.174498 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:59.174536 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:59.175025 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:59.175025 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:59.175027 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:16:59.175078 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:59.175079 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.175949 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.175950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:59.177109 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:59.177113 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.177955 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.177956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:59.178045 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:59.178059 (+ 14us) client_negotiation.cc:769] Sending connection context
0811 05:16:59.178130 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 05:16:59.178194 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"mutex_wait_us":20,"thread_start_us":113,"threads_started":1}
I20260811 05:16:59.178294 28348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:59.174632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42260 (local address 127.28.143.129:42875)
0811 05:16:59.174805 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:59.174809 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:59.174818 (+ 9us) server_negotiation.cc:407] Connection header received
0811 05:16:59.174853 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:59.174855 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:59.174869 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:59.174938 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:16:59.175150 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.175851 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.177204 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.177872 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.177941 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:59.178015 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:59.178152 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 05:16:59.178197 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":95,"threads_started":1}
I20260811 05:16:59.178671 28338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:59.174105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.131:39545 (local address 127.0.0.1:48658)
0811 05:16:59.174590 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:59.174601 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:59.174670 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:59.174953 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:59.174956 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:59.174959 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:16:59.175054 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:59.175056 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.176414 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.176415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:59.177459 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:59.177461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.178424 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.178426 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:59.178491 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:59.178495 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:16:59.178556 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 05:16:59.178602 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"mutex_wait_us":31,"thread_start_us":109,"threads_started":1}
I20260811 05:16:59.178728 28346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:59.174376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48658 (local address 127.28.143.131:39545)
0811 05:16:59.174599 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:59.174604 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:59.174678 (+ 74us) server_negotiation.cc:407] Connection header received
0811 05:16:59.174806 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:59.174809 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:59.174828 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:59.175136 (+ 308us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:16:59.175217 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.176295 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.177558 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.178305 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.178421 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:59.178497 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:59.178593 (+ 96us) server_negotiation.cc:299] Negotiation successful
0811 05:16:59.178636 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":127,"threads_started":1}
I20260811 05:16:59.179188 27730 tablet_service.cc:1467] Processing CreateTablet for tablet 36a1904a57fc42bb9e793fdff5e2c767 (DEFAULT_TABLE table=test-table [id=aaf957218bbe444d8f047c611e067efd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:16:59.179522 28199 tablet_service.cc:1467] Processing CreateTablet for tablet 36a1904a57fc42bb9e793fdff5e2c767 (DEFAULT_TABLE table=test-table [id=aaf957218bbe444d8f047c611e067efd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:16:59.179622 28344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:59.174103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:41305 (local address 127.0.0.1:39354)
0811 05:16:59.174275 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:59.174317 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:59.174386 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:59.175008 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:59.175009 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:59.175011 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:16:59.175061 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:59.175063 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.176419 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.176420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:59.177519 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:59.177523 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.178997 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.178998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:59.179452 (+ 454us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:59.179457 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 05:16:59.179507 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 05:16:59.179552 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":31,"thread_start_us":70,"threads_started":1}
I20260811 05:16:59.179651 27730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36a1904a57fc42bb9e793fdff5e2c767. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:16:59.179848 28199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36a1904a57fc42bb9e793fdff5e2c767. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:16:59.179879 28347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:59.174395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39354 (local address 127.28.143.130:41305)
0811 05:16:59.174683 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:59.174687 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:59.174697 (+ 10us) server_negotiation.cc:407] Connection header received
0811 05:16:59.174736 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:59.174739 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:59.174755 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:59.174918 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:16:59.175132 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.176295 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.177604 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.178308 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.179003 (+ 695us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:59.179078 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:59.179735 (+ 657us) server_negotiation.cc:299] Negotiation successful
0811 05:16:59.179803 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":109,"threads_started":1}
I20260811 05:16:59.182837 28352 tablet_bootstrap.cc:492] T 36a1904a57fc42bb9e793fdff5e2c767 P efe92987c46840cfbc53d2114b8533f5: Bootstrap starting.
I20260811 05:16:59.182971 28351 tablet_bootstrap.cc:492] T 36a1904a57fc42bb9e793fdff5e2c767 P 783a644a6c7c4088a5b6edd32aaed233: Bootstrap starting.
I20260811 05:16:59.183403 27957 tablet_service.cc:1467] Processing CreateTablet for tablet 36a1904a57fc42bb9e793fdff5e2c767 (DEFAULT_TABLE table=test-table [id=aaf957218bbe444d8f047c611e067efd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:16:59.183619 28351 tablet_bootstrap.cc:654] T 36a1904a57fc42bb9e793fdff5e2c767 P 783a644a6c7c4088a5b6edd32aaed233: Neither blocks nor log segments found. Creating new log.
I20260811 05:16:59.183619 28352 tablet_bootstrap.cc:654] T 36a1904a57fc42bb9e793fdff5e2c767 P efe92987c46840cfbc53d2114b8533f5: Neither blocks nor log segments found. Creating new log.
I20260811 05:16:59.183761 27957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36a1904a57fc42bb9e793fdff5e2c767. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:16:59.184223 28352 log.cc:824] T 36a1904a57fc42bb9e793fdff5e2c767 P efe92987c46840cfbc53d2114b8533f5: Log is configured to *not* fsync() on all Append() calls
I20260811 05:16:59.184223 28351 log.cc:824] T 36a1904a57fc42bb9e793fdff5e2c767 P 783a644a6c7c4088a5b6edd32aaed233: Log is configured to *not* fsync() on all Append() calls
I20260811 05:16:59.185089 28352 tablet_bootstrap.cc:492] T 36a1904a57fc42bb9e793fdff5e2c767 P efe92987c46840cfbc53d2114b8533f5: No bootstrap required, opened a new log
I20260811 05:16:59.185099 28351 tablet_bootstrap.cc:492] T 36a1904a57fc42bb9e793fdff5e2c767 P 783a644a6c7c4088a5b6edd32aaed233: No bootstrap required, opened a new log
I20260811 05:16:59.185164 28351 ts_tablet_manager.cc:1397] T 36a1904a57fc42bb9e793fdff5e2c767 P 783a644a6c7c4088a5b6edd32aaed233: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 05:16:59.185164 28352 ts_tablet_manager.cc:1397] T 36a1904a57fc42bb9e793fdff5e2c767 P efe92987c46840cfbc53d2114b8533f5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 05:16:59.186800 28356 tablet_bootstrap.cc:492] T 36a1904a57fc42bb9e793fdff5e2c767 P 816585fd4c594406b66bf2603953639b: Bootstrap starting.
I20260811 05:16:59.187099 28351 raft_consensus.cc:348] T 36a1904a57fc42bb9e793fdff5e2c767 P 783a644a6c7c4088a5b6edd32aaed233 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe92987c46840cfbc53d2114b8533f5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 42875 } } peers { permanent_uuid: "816585fd4c594406b66bf2603953639b" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41305 } } peers { permanent_uuid: "783a644a6c7c4088a5b6edd32aaed233" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 39545 } }
I20260811 05:16:59.187189 28352 raft_consensus.cc:348] T 36a1904a57fc42bb9e793fdff5e2c767 P efe92987c46840cfbc53d2114b8533f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe92987c46840cfbc53d2114b8533f5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 42875 } } peers { permanent_uuid: "816585fd4c594406b66bf2603953639b" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41305 } } peers { permanent_uuid: "783a644a6c7c4088a5b6edd32aaed233" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 39545 } }
I20260811 05:16:59.187301 28351 raft_consensus.cc:374] T 36a1904a57fc42bb9e793fdff5e2c767 P 783a644a6c7c4088a5b6edd32aaed233 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:16:59.187332 28352 raft_consensus.cc:374] T 36a1904a57fc42bb9e793fdff5e2c767 P efe92987c46840cfbc53d2114b8533f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:16:59.187352 28351 raft_consensus.cc:729] T 36a1904a57fc42bb9e793fdff5e2c767 P 783a644a6c7c4088a5b6edd32aaed233 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 783a644a6c7c4088a5b6edd32aaed233, State: Initialized, Role: FOLLOWER
I20260811 05:16:59.187366 28352 raft_consensus.cc:729] T 36a1904a57fc42bb9e793fdff5e2c767 P efe92987c46840cfbc53d2114b8533f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe92987c46840cfbc53d2114b8533f5, State: Initialized, Role: FOLLOWER
I20260811 05:16:59.187423 28356 tablet_bootstrap.cc:654] T 36a1904a57fc42bb9e793fdff5e2c767 P 816585fd4c594406b66bf2603953639b: Neither blocks nor log segments found. Creating new log.
I20260811 05:16:59.187517 28352 consensus_queue.cc:260] T 36a1904a57fc42bb9e793fdff5e2c767 P efe92987c46840cfbc53d2114b8533f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe92987c46840cfbc53d2114b8533f5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 42875 } } peers { permanent_uuid: "816585fd4c594406b66bf2603953639b" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41305 } } peers { permanent_uuid: "783a644a6c7c4088a5b6edd32aaed233" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 39545 } }
I20260811 05:16:59.187517 28351 consensus_queue.cc:260] T 36a1904a57fc42bb9e793fdff5e2c767 P 783a644a6c7c4088a5b6edd32aaed233 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe92987c46840cfbc53d2114b8533f5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 42875 } } peers { permanent_uuid: "816585fd4c594406b66bf2603953639b" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41305 } } peers { permanent_uuid: "783a644a6c7c4088a5b6edd32aaed233" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 39545 } }
I20260811 05:16:59.187829 28351 ts_tablet_manager.cc:1428] T 36a1904a57fc42bb9e793fdff5e2c767 P 783a644a6c7c4088a5b6edd32aaed233: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 05:16:59.187829 28352 ts_tablet_manager.cc:1428] T 36a1904a57fc42bb9e793fdff5e2c767 P efe92987c46840cfbc53d2114b8533f5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 05:16:59.187961 28321 heartbeater.cc:503] Master 127.28.143.190:44905 was elected leader, sending a full tablet report...
I20260811 05:16:59.187970 28356 log.cc:824] T 36a1904a57fc42bb9e793fdff5e2c767 P 816585fd4c594406b66bf2603953639b: Log is configured to *not* fsync() on all Append() calls
I20260811 05:16:59.188254 27833 heartbeater.cc:503] Master 127.28.143.190:44905 was elected leader, sending a full tablet report...
I20260811 05:16:59.188923 28356 tablet_bootstrap.cc:492] T 36a1904a57fc42bb9e793fdff5e2c767 P 816585fd4c594406b66bf2603953639b: No bootstrap required, opened a new log
I20260811 05:16:59.189000 28356 ts_tablet_manager.cc:1397] T 36a1904a57fc42bb9e793fdff5e2c767 P 816585fd4c594406b66bf2603953639b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 05:16:59.190620 28356 raft_consensus.cc:348] T 36a1904a57fc42bb9e793fdff5e2c767 P 816585fd4c594406b66bf2603953639b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe92987c46840cfbc53d2114b8533f5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 42875 } } peers { permanent_uuid: "816585fd4c594406b66bf2603953639b" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41305 } } peers { permanent_uuid: "783a644a6c7c4088a5b6edd32aaed233" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 39545 } }
I20260811 05:16:59.190766 28356 raft_consensus.cc:374] T 36a1904a57fc42bb9e793fdff5e2c767 P 816585fd4c594406b66bf2603953639b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:16:59.190788 28356 raft_consensus.cc:729] T 36a1904a57fc42bb9e793fdff5e2c767 P 816585fd4c594406b66bf2603953639b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 816585fd4c594406b66bf2603953639b, State: Initialized, Role: FOLLOWER
I20260811 05:16:59.190915 28356 consensus_queue.cc:260] T 36a1904a57fc42bb9e793fdff5e2c767 P 816585fd4c594406b66bf2603953639b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe92987c46840cfbc53d2114b8533f5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 42875 } } peers { permanent_uuid: "816585fd4c594406b66bf2603953639b" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41305 } } peers { permanent_uuid: "783a644a6c7c4088a5b6edd32aaed233" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 39545 } }
I20260811 05:16:59.191186 28075 heartbeater.cc:503] Master 127.28.143.190:44905 was elected leader, sending a full tablet report...
I20260811 05:16:59.191171 28356 ts_tablet_manager.cc:1428] T 36a1904a57fc42bb9e793fdff5e2c767 P 816585fd4c594406b66bf2603953639b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260811 05:16:59.220153 28077 tablet.cc:2369] T 36a1904a57fc42bb9e793fdff5e2c767 P 816585fd4c594406b66bf2603953639b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:16:59.228111 28358 raft_consensus.cc:482] T 36a1904a57fc42bb9e793fdff5e2c767 P 783a644a6c7c4088a5b6edd32aaed233 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:16:59.228274 28358 raft_consensus.cc:504] T 36a1904a57fc42bb9e793fdff5e2c767 P 783a644a6c7c4088a5b6edd32aaed233 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe92987c46840cfbc53d2114b8533f5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 42875 } } peers { permanent_uuid: "816585fd4c594406b66bf2603953639b" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41305 } } peers { permanent_uuid: "783a644a6c7c4088a5b6edd32aaed233" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 39545 } }
I20260811 05:16:59.228693 28358 leader_election.cc:288] T 36a1904a57fc42bb9e793fdff5e2c767 P 783a644a6c7c4088a5b6edd32aaed233 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efe92987c46840cfbc53d2114b8533f5 (127.28.143.129:42875), 816585fd4c594406b66bf2603953639b (127.28.143.130:41305)
I20260811 05:16:59.231603 28347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:59.229012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:40221 (local address 127.28.143.130:41305)
0811 05:16:59.229061 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:59.229063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:59.229214 (+ 151us) server_negotiation.cc:407] Connection header received
0811 05:16:59.229246 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:59.229247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:59.229258 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:59.229339 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:16:59.229490 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.230008 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.230917 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.231283 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.231348 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:59.231357 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:59.231491 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 05:16:59.231532 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 05:16:59.231619 28365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:59.228924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:41305 (local address 127.28.143.131:40221)
0811 05:16:59.229103 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:59.229111 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:59.229163 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:59.229340 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:59.229341 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:59.229342 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:16:59.229410 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:59.229412 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.230088 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.230089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:59.230821 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:59.230823 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.231359 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.231361 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:59.231429 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:59.231433 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:16:59.231483 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 05:16:59.231528 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":114,"threads_started":1}
I20260811 05:16:59.231719 28348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:59.229018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:50993 (local address 127.28.143.129:42875)
0811 05:16:59.229065 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:59.229067 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:59.229075 (+ 8us) server_negotiation.cc:407] Connection header received
0811 05:16:59.229116 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:59.229117 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:59.229132 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:59.229186 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:16:59.229404 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.229922 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.230890 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.231275 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.231341 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:59.231352 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:59.231611 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 05:16:59.231657 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 05:16:59.231882 28327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:59.228875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:42875 (local address 127.28.143.131:50993)
0811 05:16:59.228929 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:59.228943 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:59.229002 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:59.229239 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:59.229240 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:59.229243 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:16:59.229323 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:59.229325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.230005 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.230006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:59.230789 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:59.230791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.231430 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.231430 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:59.231527 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:59.231530 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 05:16:59.231574 (+ 44us) client_negotiation.cc:240] Negotiation successful
0811 05:16:59.231607 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260811 05:16:59.232067 27997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36a1904a57fc42bb9e793fdff5e2c767" candidate_uuid: "783a644a6c7c4088a5b6edd32aaed233" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "816585fd4c594406b66bf2603953639b" is_pre_election: true
I20260811 05:16:59.232254 27997 raft_consensus.cc:2454] T 36a1904a57fc42bb9e793fdff5e2c767 P 816585fd4c594406b66bf2603953639b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 783a644a6c7c4088a5b6edd32aaed233 in term 0.
I20260811 05:16:59.232235 27758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36a1904a57fc42bb9e793fdff5e2c767" candidate_uuid: "783a644a6c7c4088a5b6edd32aaed233" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe92987c46840cfbc53d2114b8533f5" is_pre_election: true
I20260811 05:16:59.232362 27758 raft_consensus.cc:2454] T 36a1904a57fc42bb9e793fdff5e2c767 P efe92987c46840cfbc53d2114b8533f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 783a644a6c7c4088a5b6edd32aaed233 in term 0.
I20260811 05:16:59.232585 28115 leader_election.cc:302] T 36a1904a57fc42bb9e793fdff5e2c767 P 783a644a6c7c4088a5b6edd32aaed233 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 783a644a6c7c4088a5b6edd32aaed233, 816585fd4c594406b66bf2603953639b; no voters:
I20260811 05:16:59.232761 28358 raft_consensus.cc:2781] T 36a1904a57fc42bb9e793fdff5e2c767 P 783a644a6c7c4088a5b6edd32aaed233 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:16:59.232805 28358 raft_consensus.cc:482] T 36a1904a57fc42bb9e793fdff5e2c767 P 783a644a6c7c4088a5b6edd32aaed233 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:16:59.232827 28358 raft_consensus.cc:3037] T 36a1904a57fc42bb9e793fdff5e2c767 P 783a644a6c7c4088a5b6edd32aaed233 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:16:59.233544 28358 raft_consensus.cc:504] T 36a1904a57fc42bb9e793fdff5e2c767 P 783a644a6c7c4088a5b6edd32aaed233 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe92987c46840cfbc53d2114b8533f5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 42875 } } peers { permanent_uuid: "816585fd4c594406b66bf2603953639b" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41305 } } peers { permanent_uuid: "783a644a6c7c4088a5b6edd32aaed233" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 39545 } }
I20260811 05:16:59.233671 28358 leader_election.cc:288] T 36a1904a57fc42bb9e793fdff5e2c767 P 783a644a6c7c4088a5b6edd32aaed233 [CANDIDATE]: Term 1 election: Requested vote from peers efe92987c46840cfbc53d2114b8533f5 (127.28.143.129:42875), 816585fd4c594406b66bf2603953639b (127.28.143.130:41305)
I20260811 05:16:59.233876 27997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36a1904a57fc42bb9e793fdff5e2c767" candidate_uuid: "783a644a6c7c4088a5b6edd32aaed233" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "816585fd4c594406b66bf2603953639b"
I20260811 05:16:59.233976 27997 raft_consensus.cc:3037] T 36a1904a57fc42bb9e793fdff5e2c767 P 816585fd4c594406b66bf2603953639b [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:16:59.234275 27758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36a1904a57fc42bb9e793fdff5e2c767" candidate_uuid: "783a644a6c7c4088a5b6edd32aaed233" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe92987c46840cfbc53d2114b8533f5"
I20260811 05:16:59.234359 27758 raft_consensus.cc:3037] T 36a1904a57fc42bb9e793fdff5e2c767 P efe92987c46840cfbc53d2114b8533f5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:16:59.234814 27997 raft_consensus.cc:2454] T 36a1904a57fc42bb9e793fdff5e2c767 P 816585fd4c594406b66bf2603953639b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 783a644a6c7c4088a5b6edd32aaed233 in term 1.
I20260811 05:16:59.235016 28115 leader_election.cc:302] T 36a1904a57fc42bb9e793fdff5e2c767 P 783a644a6c7c4088a5b6edd32aaed233 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 783a644a6c7c4088a5b6edd32aaed233, 816585fd4c594406b66bf2603953639b; no voters:
I20260811 05:16:59.235123 28358 raft_consensus.cc:2781] T 36a1904a57fc42bb9e793fdff5e2c767 P 783a644a6c7c4088a5b6edd32aaed233 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:16:59.235209 27758 raft_consensus.cc:2454] T 36a1904a57fc42bb9e793fdff5e2c767 P efe92987c46840cfbc53d2114b8533f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 783a644a6c7c4088a5b6edd32aaed233 in term 1.
I20260811 05:16:59.235301 28358 raft_consensus.cc:686] T 36a1904a57fc42bb9e793fdff5e2c767 P 783a644a6c7c4088a5b6edd32aaed233 [term 1 LEADER]: Becoming Leader. State: Replica: 783a644a6c7c4088a5b6edd32aaed233, State: Running, Role: LEADER
I20260811 05:16:59.235399 28358 consensus_queue.cc:237] T 36a1904a57fc42bb9e793fdff5e2c767 P 783a644a6c7c4088a5b6edd32aaed233 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe92987c46840cfbc53d2114b8533f5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 42875 } } peers { permanent_uuid: "816585fd4c594406b66bf2603953639b" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41305 } } peers { permanent_uuid: "783a644a6c7c4088a5b6edd32aaed233" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 39545 } }
I20260811 05:16:59.236253 27488 catalog_manager.cc:5495] T 36a1904a57fc42bb9e793fdff5e2c767 P 783a644a6c7c4088a5b6edd32aaed233 reported cstate change: term changed from 0 to 1, leader changed from <none> to 783a644a6c7c4088a5b6edd32aaed233 (127.28.143.131). New cstate: current_term: 1 leader_uuid: "783a644a6c7c4088a5b6edd32aaed233" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efe92987c46840cfbc53d2114b8533f5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 42875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "816585fd4c594406b66bf2603953639b" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "783a644a6c7c4088a5b6edd32aaed233" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 39545 } health_report { overall_health: HEALTHY } } }
I20260811 05:16:59.253247 28346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:59.250113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48668 (local address 127.28.143.131:39545)
0811 05:16:59.250175 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:59.250177 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:59.250187 (+ 10us) server_negotiation.cc:407] Connection header received
0811 05:16:59.250233 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:59.250235 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:59.250261 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:59.250342 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:16:59.250547 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.251259 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.252379 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.252594 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.252706 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:59.252870 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:16:59.252997 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:16:59.253073 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:59.253109 (+ 36us) server_negotiation.cc:299] Negotiation successful
0811 05:16:59.253134 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 05:16:59.255131 27758 raft_consensus.cc:1261] T 36a1904a57fc42bb9e793fdff5e2c767 P efe92987c46840cfbc53d2114b8533f5 [term 1 FOLLOWER]: Refusing update from remote peer 783a644a6c7c4088a5b6edd32aaed233: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:16:59.255182 27997 raft_consensus.cc:1261] T 36a1904a57fc42bb9e793fdff5e2c767 P 816585fd4c594406b66bf2603953639b [term 1 FOLLOWER]: Refusing update from remote peer 783a644a6c7c4088a5b6edd32aaed233: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:16:59.255489 28366 consensus_queue.cc:1035] T 36a1904a57fc42bb9e793fdff5e2c767 P 783a644a6c7c4088a5b6edd32aaed233 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe92987c46840cfbc53d2114b8533f5" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 42875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:16:59.255631 28358 consensus_queue.cc:1035] T 36a1904a57fc42bb9e793fdff5e2c767 P 783a644a6c7c4088a5b6edd32aaed233 [LEADER]: Connected to new peer: Peer: permanent_uuid: "816585fd4c594406b66bf2603953639b" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 41305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:16:59.258580 28377 mvcc.cc:205] Tried to move back new op lower bound from 7317198517266337792 to 7317198517188870144. Current Snapshot: MvccSnapshot[applied={T|T < 7317198517266337792}]
I20260811 05:16:59.258764 28374 mvcc.cc:205] Tried to move back new op lower bound from 7317198517266337792 to 7317198517188870144. Current Snapshot: MvccSnapshot[applied={T|T < 7317198517266337792}]
I20260811 05:16:59.260280 27488 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34582:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:16:59.260371 27488 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34582:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:16:59.261185 27488 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9114e391bc2c4e8d8444d94da2367c4e: Sending DeleteTablet for 3 replicas of tablet 36a1904a57fc42bb9e793fdff5e2c767
I20260811 05:16:59.261572 27957 tablet_service.cc:1514] Processing DeleteTablet for tablet 36a1904a57fc42bb9e793fdff5e2c767 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:16:59 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:39354
I20260811 05:16:59.261618 27730 tablet_service.cc:1514] Processing DeleteTablet for tablet 36a1904a57fc42bb9e793fdff5e2c767 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:16:59 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:42260
I20260811 05:16:59.261706 28199 tablet_service.cc:1514] Processing DeleteTablet for tablet 36a1904a57fc42bb9e793fdff5e2c767 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:16:59 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:48658
I20260811 05:16:59.261976 28390 tablet_replica.cc:330] T 36a1904a57fc42bb9e793fdff5e2c767 P 816585fd4c594406b66bf2603953639b: stopping tablet replica
I20260811 05:16:59.262109 28390 raft_consensus.cc:2229] T 36a1904a57fc42bb9e793fdff5e2c767 P 816585fd4c594406b66bf2603953639b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:16:59.262231 28390 raft_consensus.cc:2258] T 36a1904a57fc42bb9e793fdff5e2c767 P 816585fd4c594406b66bf2603953639b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:16:59.262245 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 27607
I20260811 05:16:59.262732 28390 ts_tablet_manager.cc:1905] T 36a1904a57fc42bb9e793fdff5e2c767 P 816585fd4c594406b66bf2603953639b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:16:59.265033 28390 ts_tablet_manager.cc:1918] T 36a1904a57fc42bb9e793fdff5e2c767 P 816585fd4c594406b66bf2603953639b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:16:59.265134 28390 log.cc:1196] T 36a1904a57fc42bb9e793fdff5e2c767 P 816585fd4c594406b66bf2603953639b: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-1/wal/wals/36a1904a57fc42bb9e793fdff5e2c767
I20260811 05:16:59.265470 28390 ts_tablet_manager.cc:1939] T 36a1904a57fc42bb9e793fdff5e2c767 P 816585fd4c594406b66bf2603953639b: Deleting consensus metadata
I20260811 05:16:59.266098 27458 catalog_manager.cc:4841] TS 816585fd4c594406b66bf2603953639b (127.28.143.130:41305): tablet 36a1904a57fc42bb9e793fdff5e2c767 (table test-table [id=aaf957218bbe444d8f047c611e067efd]) successfully deleted
I20260811 05:16:59.276683 28393 tablet_replica.cc:330] T 36a1904a57fc42bb9e793fdff5e2c767 P 783a644a6c7c4088a5b6edd32aaed233: stopping tablet replica
I20260811 05:16:59.276849 28393 raft_consensus.cc:2229] T 36a1904a57fc42bb9e793fdff5e2c767 P 783a644a6c7c4088a5b6edd32aaed233 [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:16:59.276985 28393 raft_consensus.cc:2258] T 36a1904a57fc42bb9e793fdff5e2c767 P 783a644a6c7c4088a5b6edd32aaed233 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:16:59.277632 28393 ts_tablet_manager.cc:1905] T 36a1904a57fc42bb9e793fdff5e2c767 P 783a644a6c7c4088a5b6edd32aaed233: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:16:59.279429 28393 ts_tablet_manager.cc:1918] T 36a1904a57fc42bb9e793fdff5e2c767 P 783a644a6c7c4088a5b6edd32aaed233: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:16:59.279524 28393 log.cc:1196] T 36a1904a57fc42bb9e793fdff5e2c767 P 783a644a6c7c4088a5b6edd32aaed233: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425159060450-29246-0/minicluster-data/ts-2/wal/wals/36a1904a57fc42bb9e793fdff5e2c767
I20260811 05:16:59.279852 28393 ts_tablet_manager.cc:1939] T 36a1904a57fc42bb9e793fdff5e2c767 P 783a644a6c7c4088a5b6edd32aaed233: Deleting consensus metadata
I20260811 05:16:59.280355 27460 catalog_manager.cc:4841] TS 783a644a6c7c4088a5b6edd32aaed233 (127.28.143.131:39545): tablet 36a1904a57fc42bb9e793fdff5e2c767 (table test-table [id=aaf957218bbe444d8f047c611e067efd]) successfully deleted
W20260811 05:16:59.284777 27462 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.143.129:42875 (error 108)
I20260811 05:16:59.284992 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 27842
I20260811 05:16:59.285424 28341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:59.285284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:42875 (local address 127.0.0.1:42268)
0811 05:16:59.285340 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:59.285374 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.129:42875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260811 05:16:59.285562 27462 catalog_manager.cc:4584] TS efe92987c46840cfbc53d2114b8533f5 (127.28.143.129:42875): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 36a1904a57fc42bb9e793fdff5e2c767: Network error: Client connection negotiation failed: client connection to 127.28.143.129:42875: connect: Connection refused (error 111)
I20260811 05:16:59.303416 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 28082
I20260811 05:16:59.318744 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 27429
2026-08-11T05:16:59Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2241 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T05:16:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:16:59Z Disabled control of system clock
I20260811 05:16:59.360597 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:35677
--webserver_interface=127.28.143.190
--webserver_port=0
--builtin_ntp_servers=127.28.143.148:38281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:35677
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 05:16:59.446599 28404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:16:59.446911 28404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:16:59.446956 28404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:16:59.448523 28404 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:16:59.448652 28404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:16:59.448674 28404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:16:59.448693 28404 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:16:59.448710 28404 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:16:59.450615 28404 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:38281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:35677
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:35677
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:16:59.450976 28404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:16:59.451332 28404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:16:59.456969 28412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:59.457002 28414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:59.456964 28419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:59.457360 28404 server_base.cc:1034] running on GCE node
I20260811 05:16:59.457695 28404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:16:59.458026 28404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:16:59.459179 28404 hybrid_clock.cc:648] HybridClock initialized: now 1786425419459163 us; error 40 us; skew 500 ppm
I20260811 05:16:59.460850 28404 webserver.cc:460] Webserver started at http://127.28.143.190:40551/ using document root <none> and password file <none>
I20260811 05:16:59.461134 28404 fs_manager.cc:359] Metadata directory not provided
I20260811 05:16:59.461194 28404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:16:59.461342 28404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:16:59.462527 28404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "40c182c04ee447e6919cad34ff78baca"
format_stamp: "Formatted at 2026-08-11 05:16:59 on dist-test-slave-f7th"
server_key: "c5e456923270e66d4c2de90fc50cfe40"
server_key_iv: "a6b52933210841074802841404d5459b"
server_key_version: "encryptionkey@0"
I20260811 05:16:59.462909 28404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "40c182c04ee447e6919cad34ff78baca"
format_stamp: "Formatted at 2026-08-11 05:16:59 on dist-test-slave-f7th"
server_key: "c5e456923270e66d4c2de90fc50cfe40"
server_key_iv: "a6b52933210841074802841404d5459b"
server_key_version: "encryptionkey@0"
I20260811 05:16:59.464491 28404 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 05:16:59.465593 28428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:59.465992 28404 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:16:59.466187 28404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "40c182c04ee447e6919cad34ff78baca"
format_stamp: "Formatted at 2026-08-11 05:16:59 on dist-test-slave-f7th"
server_key: "c5e456923270e66d4c2de90fc50cfe40"
server_key_iv: "a6b52933210841074802841404d5459b"
server_key_version: "encryptionkey@0"
I20260811 05:16:59.466289 28404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:16:59.485834 28404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:16:59.486271 28404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:16:59.486440 28404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:16:59.490751 28404 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:35677
I20260811 05:16:59.490826 28532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:35677 every 8 connection(s)
I20260811 05:16:59.491173 28404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:16:59.492223 28533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:16:59.494776 28533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40c182c04ee447e6919cad34ff78baca: Bootstrap starting.
I20260811 05:16:59.495428 28533 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 40c182c04ee447e6919cad34ff78baca: Neither blocks nor log segments found. Creating new log.
I20260811 05:16:59.495739 28533 log.cc:824] T 00000000000000000000000000000000 P 40c182c04ee447e6919cad34ff78baca: Log is configured to *not* fsync() on all Append() calls
I20260811 05:16:59.496554 28533 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40c182c04ee447e6919cad34ff78baca: No bootstrap required, opened a new log
I20260811 05:16:59.496745 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 28404
I20260811 05:16:59.496847 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:16:59.497051 29246 external_mini_cluster.cc:1428] Setting key efce7cb8185acc476607c325ef26d46a
I20260811 05:16:59.498576 28533 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40c182c04ee447e6919cad34ff78baca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c182c04ee447e6919cad34ff78baca" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 35677 } }
I20260811 05:16:59.498720 28533 raft_consensus.cc:374] T 00000000000000000000000000000000 P 40c182c04ee447e6919cad34ff78baca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:16:59.498754 28533 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40c182c04ee447e6919cad34ff78baca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40c182c04ee447e6919cad34ff78baca, State: Initialized, Role: FOLLOWER
I20260811 05:16:59.498912 28533 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40c182c04ee447e6919cad34ff78baca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c182c04ee447e6919cad34ff78baca" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 35677 } }
I20260811 05:16:59.498986 28533 raft_consensus.cc:388] T 00000000000000000000000000000000 P 40c182c04ee447e6919cad34ff78baca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:16:59.499017 28533 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40c182c04ee447e6919cad34ff78baca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:16:59.499068 28533 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40c182c04ee447e6919cad34ff78baca [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:16:59.499658 28533 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40c182c04ee447e6919cad34ff78baca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c182c04ee447e6919cad34ff78baca" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 35677 } }
I20260811 05:16:59.499761 28533 leader_election.cc:302] T 00000000000000000000000000000000 P 40c182c04ee447e6919cad34ff78baca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40c182c04ee447e6919cad34ff78baca; no voters:
I20260811 05:16:59.499939 28533 leader_election.cc:288] T 00000000000000000000000000000000 P 40c182c04ee447e6919cad34ff78baca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:16:59.499982 28540 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40c182c04ee447e6919cad34ff78baca [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:16:59.500140 28540 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40c182c04ee447e6919cad34ff78baca [term 1 LEADER]: Becoming Leader. State: Replica: 40c182c04ee447e6919cad34ff78baca, State: Running, Role: LEADER
I20260811 05:16:59.500231 28533 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40c182c04ee447e6919cad34ff78baca [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:16:59.500286 28540 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40c182c04ee447e6919cad34ff78baca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c182c04ee447e6919cad34ff78baca" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 35677 } }
I20260811 05:16:59.500722 28541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40c182c04ee447e6919cad34ff78baca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40c182c04ee447e6919cad34ff78baca. Latest consensus state: current_term: 1 leader_uuid: "40c182c04ee447e6919cad34ff78baca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c182c04ee447e6919cad34ff78baca" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 35677 } } }
I20260811 05:16:59.500818 28541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40c182c04ee447e6919cad34ff78baca [sys.catalog]: This master's current role is: LEADER
I20260811 05:16:59.501070 28551 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:16:59.501689 28551 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:16:59.502040 28542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40c182c04ee447e6919cad34ff78baca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "40c182c04ee447e6919cad34ff78baca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c182c04ee447e6919cad34ff78baca" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 35677 } } }
I20260811 05:16:59.502141 28542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40c182c04ee447e6919cad34ff78baca [sys.catalog]: This master's current role is: LEADER
I20260811 05:16:59.503166 28565 mvcc.cc:205] Tried to move back new op lower bound from 7317198518282821632 to 7317198518273765376. Current Snapshot: MvccSnapshot[applied={T|T < 7317198518282821632}]
I20260811 05:16:59.503631 28551 catalog_manager.cc:1347] Generated new cluster ID: 3f5e07ec16f645dcb4925ff75ea5c707
I20260811 05:16:59.503680 28551 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:16:59.504165 28539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:59.497613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49442 (local address 127.28.143.190:35677)
0811 05:16:59.497833 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:59.497841 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:59.497867 (+ 26us) server_negotiation.cc:407] Connection header received
0811 05:16:59.498544 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:59.498551 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:59.498617 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:59.498746 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:59.498954 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.500508 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.501566 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.502156 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.502276 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:59.503327 (+ 1051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:59.503332 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:59.503332 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:16:59.503541 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:59.503587 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:16:59.503603 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:59.503670 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:59.503889 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 05:16:59.503969 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":84,"threads_started":1}
I20260811 05:16:59.531903 28551 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:16:59.532822 28551 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:16:59.538542 28551 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 40c182c04ee447e6919cad34ff78baca: Generated new TSK 0
I20260811 05:16:59.538976 28551 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:16:59.544787 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.129:0
--local_ip_for_outbound_sockets=127.28.143.129
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:35677
--builtin_ntp_servers=127.28.143.148:38281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 05:16:59.631633 28573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:16:59.631862 28573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:16:59.631886 28573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:16:59.633208 28573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:16:59.633258 28573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:16:59.633287 28573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.129
I20260811 05:16:59.634702 28573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:38281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:35677
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.129
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:16:59.634943 28573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:16:59.635257 28573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:16:59.640443 28579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:59.640522 28573 server_base.cc:1034] running on GCE node
W20260811 05:16:59.640537 28580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:59.640426 28583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:59.640843 28573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:16:59.641132 28573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:16:59.642323 28573 hybrid_clock.cc:648] HybridClock initialized: now 1786425419642304 us; error 34 us; skew 500 ppm
I20260811 05:16:59.643656 28573 webserver.cc:460] Webserver started at http://127.28.143.129:38367/ using document root <none> and password file <none>
I20260811 05:16:59.643916 28573 fs_manager.cc:359] Metadata directory not provided
I20260811 05:16:59.643975 28573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:16:59.644116 28573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:16:59.645268 28573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-0/data/instance:
uuid: "c9c2f9de568c4ceaa17d42fac0ec64f7"
format_stamp: "Formatted at 2026-08-11 05:16:59 on dist-test-slave-f7th"
server_key: "73be1c38562f65de66bfcf66ea28bbe8"
server_key_iv: "c21d90de73795d0282bccd88ca069b1f"
server_key_version: "encryptionkey@0"
I20260811 05:16:59.645624 28573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance:
uuid: "c9c2f9de568c4ceaa17d42fac0ec64f7"
format_stamp: "Formatted at 2026-08-11 05:16:59 on dist-test-slave-f7th"
server_key: "73be1c38562f65de66bfcf66ea28bbe8"
server_key_iv: "c21d90de73795d0282bccd88ca069b1f"
server_key_version: "encryptionkey@0"
I20260811 05:16:59.647050 28573 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 05:16:59.647836 28595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:59.648011 28573 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 05:16:59.648105 28573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-0/wal
uuid: "c9c2f9de568c4ceaa17d42fac0ec64f7"
format_stamp: "Formatted at 2026-08-11 05:16:59 on dist-test-slave-f7th"
server_key: "73be1c38562f65de66bfcf66ea28bbe8"
server_key_iv: "c21d90de73795d0282bccd88ca069b1f"
server_key_version: "encryptionkey@0"
I20260811 05:16:59.648169 28573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:16:59.656455 28573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:16:59.656823 28573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:16:59.657003 28573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:16:59.657243 28573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:16:59.657672 28573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:16:59.657713 28573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:59.657745 28573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:16:59.657763 28573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:59.663415 28573 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.129:42715
I20260811 05:16:59.663481 28796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.129:42715 every 8 connection(s)
I20260811 05:16:59.663805 28573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
I20260811 05:16:59.669118 28539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:59.664577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:52939 (local address 127.28.143.190:35677)
0811 05:16:59.664661 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:59.664664 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:59.664888 (+ 224us) server_negotiation.cc:407] Connection header received
0811 05:16:59.665417 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:59.665419 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:59.665435 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:59.665504 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:59.666227 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.666735 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.667719 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.667917 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.667975 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:59.668857 (+ 882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:59.668859 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:59.668859 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:16:59.668865 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:59.668882 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:16:59.668886 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:59.668934 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:59.668998 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 05:16:59.669051 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 05:16:59.669276 28802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:59.664521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:35677 (local address 127.28.143.129:52939)
0811 05:16:59.664759 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:59.664804 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:59.665295 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:59.665583 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:59.665587 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:59.665596 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:16:59.666125 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:59.666130 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.666819 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.666820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:59.667628 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:59.667630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.667989 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.667989 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:59.668109 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:59.668714 (+ 605us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:16:59.668715 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:16:59.668747 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:16:59.668775 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:16:59.668941 (+ 166us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:16:59.668944 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:16:59.668947 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 05:16:59.668993 (+ 46us) client_negotiation.cc:240] Negotiation successful
0811 05:16:59.669083 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":126,"threads_started":1}
I20260811 05:16:59.669778 28798 heartbeater.cc:347] Connected to a master server at 127.28.143.190:35677
I20260811 05:16:59.669879 28798 heartbeater.cc:464] Registering TS with master...
I20260811 05:16:59.670048 28798 heartbeater.cc:511] Master 127.28.143.190:35677 requested a full tablet report, sending...
I20260811 05:16:59.670559 28452 ts_manager.cc:194] Registered new tserver with Master: c9c2f9de568c4ceaa17d42fac0ec64f7 (127.28.143.129:42715)
I20260811 05:16:59.670972 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 28573
I20260811 05:16:59.671068 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance
I20260811 05:16:59.671255 29246 external_mini_cluster.cc:1428] Setting key 599436127c054ff44c95e54cc00291c2
I20260811 05:16:59.671241 28452 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.143.129:52939
I20260811 05:16:59.672231 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.130:0
--local_ip_for_outbound_sockets=127.28.143.130
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:35677
--builtin_ntp_servers=127.28.143.148:38281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 05:16:59.764174 28804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:16:59.764415 28804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:16:59.764439 28804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:16:59.765792 28804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:16:59.765857 28804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:16:59.765885 28804 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.130
I20260811 05:16:59.767485 28804 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:38281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:35677
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28804
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.130
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:16:59.767875 28804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:16:59.768368 28804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:16:59.773914 28811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:59.773939 28813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:59.773962 28804 server_base.cc:1034] running on GCE node
W20260811 05:16:59.773939 28816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:59.774374 28804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:16:59.774647 28804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:16:59.775837 28804 hybrid_clock.cc:648] HybridClock initialized: now 1786425419775820 us; error 29 us; skew 500 ppm
I20260811 05:16:59.777346 28804 webserver.cc:460] Webserver started at http://127.28.143.130:46395/ using document root <none> and password file <none>
I20260811 05:16:59.777603 28804 fs_manager.cc:359] Metadata directory not provided
I20260811 05:16:59.777655 28804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:16:59.777796 28804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:16:59.778975 28804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-1/data/instance:
uuid: "42cbd08113224eb086987ae22302a626"
format_stamp: "Formatted at 2026-08-11 05:16:59 on dist-test-slave-f7th"
server_key: "bc7e6d2d52831347a49cdda87efda316"
server_key_iv: "3bac31182f8595943757532d979f374a"
server_key_version: "encryptionkey@0"
I20260811 05:16:59.779321 28804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance:
uuid: "42cbd08113224eb086987ae22302a626"
format_stamp: "Formatted at 2026-08-11 05:16:59 on dist-test-slave-f7th"
server_key: "bc7e6d2d52831347a49cdda87efda316"
server_key_iv: "3bac31182f8595943757532d979f374a"
server_key_version: "encryptionkey@0"
I20260811 05:16:59.780850 28804 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260811 05:16:59.781744 28826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:59.781934 28804 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:16:59.782034 28804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-1/wal
uuid: "42cbd08113224eb086987ae22302a626"
format_stamp: "Formatted at 2026-08-11 05:16:59 on dist-test-slave-f7th"
server_key: "bc7e6d2d52831347a49cdda87efda316"
server_key_iv: "3bac31182f8595943757532d979f374a"
server_key_version: "encryptionkey@0"
I20260811 05:16:59.782111 28804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:16:59.796856 28804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:16:59.797197 28804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:16:59.797328 28804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:16:59.797569 28804 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:16:59.798029 28804 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:16:59.798084 28804 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:59.798107 28804 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:16:59.798118 28804 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:59.803792 28804 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.130:44925
I20260811 05:16:59.803861 29028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.130:44925 every 8 connection(s)
I20260811 05:16:59.804217 28804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
I20260811 05:16:59.810103 28539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:59.805152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:43651 (local address 127.28.143.190:35677)
0811 05:16:59.805264 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:59.805266 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:59.805491 (+ 225us) server_negotiation.cc:407] Connection header received
0811 05:16:59.806073 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:59.806075 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:59.806089 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:59.806147 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:59.807006 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.807535 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.808546 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.808795 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.808858 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:59.809785 (+ 927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:59.809786 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:59.809787 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:16:59.809793 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:59.809809 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:16:59.809814 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:59.809858 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:59.809947 (+ 89us) server_negotiation.cc:299] Negotiation successful
0811 05:16:59.809984 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 05:16:59.810328 29034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:59.805038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:35677 (local address 127.28.143.130:43651)
0811 05:16:59.805333 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:59.805380 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:59.805921 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:59.806284 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:59.806289 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:59.806301 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:16:59.806881 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:59.806887 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.807636 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.807637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:59.808445 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:59.808447 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.808877 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.808878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:59.809015 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:59.809622 (+ 607us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:16:59.809622 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:16:59.809655 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:16:59.809681 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:16:59.809866 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:16:59.809872 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:16:59.809877 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 05:16:59.809942 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 05:16:59.810087 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":99,"threads_started":1}
I20260811 05:16:59.810328 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 28804
I20260811 05:16:59.810422 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance
I20260811 05:16:59.810650 29246 external_mini_cluster.cc:1428] Setting key 9654470778a9396d8eb6f78254d7893c
I20260811 05:16:59.810868 29031 heartbeater.cc:347] Connected to a master server at 127.28.143.190:35677
I20260811 05:16:59.810995 29031 heartbeater.cc:464] Registering TS with master...
I20260811 05:16:59.811198 29031 heartbeater.cc:511] Master 127.28.143.190:35677 requested a full tablet report, sending...
I20260811 05:16:59.811599 28452 ts_manager.cc:194] Registered new tserver with Master: 42cbd08113224eb086987ae22302a626 (127.28.143.130:44925)
I20260811 05:16:59.811774 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.131:0
--local_ip_for_outbound_sockets=127.28.143.131
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:35677
--builtin_ntp_servers=127.28.143.148:38281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 05:16:59.812054 28452 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.143.130:43651
W20260811 05:16:59.898759 29035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:16:59.898998 29035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:16:59.899021 29035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:16:59.900516 29035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:16:59.900583 29035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:16:59.900651 29035 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.131
I20260811 05:16:59.902251 29035 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:38281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:35677
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.131
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:16:59.902489 29035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:16:59.902781 29035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:16:59.908016 29044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:59.908018 29042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:59.908123 29035 server_base.cc:1034] running on GCE node
W20260811 05:16:59.908018 29047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:59.908464 29035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:16:59.908758 29035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:16:59.909931 29035 hybrid_clock.cc:648] HybridClock initialized: now 1786425419909913 us; error 32 us; skew 500 ppm
I20260811 05:16:59.911351 29035 webserver.cc:460] Webserver started at http://127.28.143.131:38773/ using document root <none> and password file <none>
I20260811 05:16:59.911625 29035 fs_manager.cc:359] Metadata directory not provided
I20260811 05:16:59.911681 29035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:16:59.911839 29035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:16:59.912995 29035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-2/data/instance:
uuid: "f1c5f719ec0e491b967e6bd484966f57"
format_stamp: "Formatted at 2026-08-11 05:16:59 on dist-test-slave-f7th"
server_key: "a7bfe11e1bf98900bdda4e5532383468"
server_key_iv: "45ab546aafca4bce5e0057e5a09135d3"
server_key_version: "encryptionkey@0"
I20260811 05:16:59.913347 29035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance:
uuid: "f1c5f719ec0e491b967e6bd484966f57"
format_stamp: "Formatted at 2026-08-11 05:16:59 on dist-test-slave-f7th"
server_key: "a7bfe11e1bf98900bdda4e5532383468"
server_key_iv: "45ab546aafca4bce5e0057e5a09135d3"
server_key_version: "encryptionkey@0"
I20260811 05:16:59.914618 29035 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260811 05:16:59.915324 29056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:59.915470 29035 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 05:16:59.915561 29035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-2/wal
uuid: "f1c5f719ec0e491b967e6bd484966f57"
format_stamp: "Formatted at 2026-08-11 05:16:59 on dist-test-slave-f7th"
server_key: "a7bfe11e1bf98900bdda4e5532383468"
server_key_iv: "45ab546aafca4bce5e0057e5a09135d3"
server_key_version: "encryptionkey@0"
I20260811 05:16:59.915627 29035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:16:59.927742 29035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:16:59.928081 29035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:16:59.928224 29035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:16:59.928457 29035 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:16:59.928833 29035 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:16:59.928870 29035 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:59.928902 29035 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:16:59.928921 29035 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:59.934746 29035 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.131:44515
I20260811 05:16:59.934795 29268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.131:44515 every 8 connection(s)
I20260811 05:16:59.935141 29035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
I20260811 05:16:59.938212 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 29035
I20260811 05:16:59.938297 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance
I20260811 05:16:59.938468 29246 external_mini_cluster.cc:1428] Setting key 8d95cb3431d3a32a97f0647f18121e42
I20260811 05:16:59.940542 28539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:59.935838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:57065 (local address 127.28.143.190:35677)
0811 05:16:59.935921 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:59.935923 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:59.936146 (+ 223us) server_negotiation.cc:407] Connection header received
0811 05:16:59.936630 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:59.936631 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:59.936644 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:59.936702 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:59.937356 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.937848 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.938960 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.939221 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.939292 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:59.940252 (+ 960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:59.940254 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:59.940254 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:16:59.940258 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:59.940274 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:16:59.940278 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:59.940333 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:59.940454 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 05:16:59.940481 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 05:16:59.940806 29274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:59.935824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:35677 (local address 127.28.143.131:57065)
0811 05:16:59.936026 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:59.936063 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:59.936538 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:59.936776 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:59.936779 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:59.936787 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:16:59.937293 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:59.937298 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.937911 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.937911 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:59.938886 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:59.938890 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.939346 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.939347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:59.939494 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:59.940089 (+ 595us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:16:59.940089 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:16:59.940120 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:16:59.940146 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:16:59.940368 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:16:59.940375 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:16:59.940378 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 05:16:59.940454 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 05:16:59.940542 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":92,"threads_started":1}
I20260811 05:16:59.941250 29271 heartbeater.cc:347] Connected to a master server at 127.28.143.190:35677
I20260811 05:16:59.941340 29271 heartbeater.cc:464] Registering TS with master...
I20260811 05:16:59.941494 29271 heartbeater.cc:511] Master 127.28.143.190:35677 requested a full tablet report, sending...
I20260811 05:16:59.941820 28452 ts_manager.cc:194] Registered new tserver with Master: f1c5f719ec0e491b967e6bd484966f57 (127.28.143.131:44515)
I20260811 05:16:59.942157 28452 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.143.131:57065
I20260811 05:16:59.950719 29246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:16:59.955032 28539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:59.952293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49452 (local address 127.28.143.190:35677)
0811 05:16:59.952349 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:59.952351 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:59.952438 (+ 87us) server_negotiation.cc:407] Connection header received
0811 05:16:59.952484 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:59.952485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:59.952496 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:59.952547 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:59.952707 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.953223 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.953948 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.954125 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.954171 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:59.954779 (+ 608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:59.954780 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:59.954781 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:16:59.954785 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:59.954799 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:16:59.954803 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:59.954845 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:59.954881 (+ 36us) server_negotiation.cc:299] Negotiation successful
0811 05:16:59.954934 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 05:16:59.956699 28452 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49452:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:16:59.958247 28452 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:16:59.964705 29296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:59.960737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35990 (local address 127.28.143.129:42715)
0811 05:16:59.960922 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:59.960927 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:59.961014 (+ 87us) server_negotiation.cc:407] Connection header received
0811 05:16:59.961063 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:59.961066 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:59.961081 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:59.961161 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:16:59.961380 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.962241 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.963704 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.964243 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.964307 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:59.964416 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:59.964560 (+ 144us) server_negotiation.cc:299] Negotiation successful
0811 05:16:59.964606 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":90,"threads_started":1}
I20260811 05:16:59.964692 29292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:59.960534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35116 (local address 127.28.143.131:44515)
0811 05:16:59.961434 (+ 900us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:59.961438 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:59.961449 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:16:59.961490 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:59.961492 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:59.961508 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:59.961581 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:16:59.961776 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.962756 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.963774 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.964263 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.964334 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:59.964423 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:59.964531 (+ 108us) server_negotiation.cc:299] Negotiation successful
0811 05:16:59.964571 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":107,"threads_started":1}
I20260811 05:16:59.964691 29287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:59.960443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.131:44515 (local address 127.0.0.1:35116)
0811 05:16:59.960798 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:59.960843 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:59.960911 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:59.961651 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:59.961652 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:59.961654 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:16:59.961693 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:59.961694 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.962829 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.962830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:59.963705 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:59.963708 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.964346 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.964347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:59.964429 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:59.964433 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:16:59.964512 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 05:16:59.964571 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":79,"threads_started":1}
I20260811 05:16:59.965075 29290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:59.960445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:42715 (local address 127.0.0.1:35990)
0811 05:16:59.960917 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:59.960927 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:59.961012 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:59.961200 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:59.961203 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:59.961205 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:16:59.961287 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:59.961289 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.962321 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.962322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:59.963605 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:59.963608 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.964329 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.964330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:59.964415 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:59.964430 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 05:16:59.964949 (+ 519us) client_negotiation.cc:240] Negotiation successful
0811 05:16:59.965001 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"mutex_wait_us":4,"thread_start_us":107,"threads_started":1}
I20260811 05:16:59.965644 29291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:59.960445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:44925 (local address 127.0.0.1:51490)
0811 05:16:59.961072 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:59.961079 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:59.961113 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:59.961274 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:59.961275 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:59.961278 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:16:59.961325 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:59.961327 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.962538 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.962539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:59.963605 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:59.963608 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.965383 (+ 1775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.965384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:59.965454 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:59.965459 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 05:16:59.965510 (+ 51us) client_negotiation.cc:240] Negotiation successful
0811 05:16:59.965572 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"mutex_wait_us":33,"thread_start_us":81,"threads_started":1}
I20260811 05:16:59.965610 29143 tablet_service.cc:1467] Processing CreateTablet for tablet 824263a76011420a8951bd8861f8a722 (DEFAULT_TABLE table=test-table [id=9413220d20c44d6995b0663d8be7721d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:16:59.965878 29295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:59.960737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51490 (local address 127.28.143.130:44925)
0811 05:16:59.961015 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:59.961018 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:59.961147 (+ 129us) server_negotiation.cc:407] Connection header received
0811 05:16:59.961188 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:59.961190 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:59.961204 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:59.961274 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:16:59.961377 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.962444 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.964461 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.965196 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.965266 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:59.965346 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:59.965750 (+ 404us) server_negotiation.cc:299] Negotiation successful
0811 05:16:59.965797 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":86,"threads_started":1}
I20260811 05:16:59.965790 28687 tablet_service.cc:1467] Processing CreateTablet for tablet 824263a76011420a8951bd8861f8a722 (DEFAULT_TABLE table=test-table [id=9413220d20c44d6995b0663d8be7721d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:16:59.966048 29143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 824263a76011420a8951bd8861f8a722. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:16:59.966089 28687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 824263a76011420a8951bd8861f8a722. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:16:59.969509 28913 tablet_service.cc:1467] Processing CreateTablet for tablet 824263a76011420a8951bd8861f8a722 (DEFAULT_TABLE table=test-table [id=9413220d20c44d6995b0663d8be7721d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:16:59.969910 29298 tablet_bootstrap.cc:492] T 824263a76011420a8951bd8861f8a722 P f1c5f719ec0e491b967e6bd484966f57: Bootstrap starting.
I20260811 05:16:59.969957 28913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 824263a76011420a8951bd8861f8a722. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:16:59.970686 29298 tablet_bootstrap.cc:654] T 824263a76011420a8951bd8861f8a722 P f1c5f719ec0e491b967e6bd484966f57: Neither blocks nor log segments found. Creating new log.
I20260811 05:16:59.971094 29298 log.cc:824] T 824263a76011420a8951bd8861f8a722 P f1c5f719ec0e491b967e6bd484966f57: Log is configured to *not* fsync() on all Append() calls
I20260811 05:16:59.971385 29297 tablet_bootstrap.cc:492] T 824263a76011420a8951bd8861f8a722 P c9c2f9de568c4ceaa17d42fac0ec64f7: Bootstrap starting.
I20260811 05:16:59.971805 29298 tablet_bootstrap.cc:492] T 824263a76011420a8951bd8861f8a722 P f1c5f719ec0e491b967e6bd484966f57: No bootstrap required, opened a new log
I20260811 05:16:59.971875 29298 ts_tablet_manager.cc:1397] T 824263a76011420a8951bd8861f8a722 P f1c5f719ec0e491b967e6bd484966f57: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 05:16:59.972096 29297 tablet_bootstrap.cc:654] T 824263a76011420a8951bd8861f8a722 P c9c2f9de568c4ceaa17d42fac0ec64f7: Neither blocks nor log segments found. Creating new log.
I20260811 05:16:59.972420 29297 log.cc:824] T 824263a76011420a8951bd8861f8a722 P c9c2f9de568c4ceaa17d42fac0ec64f7: Log is configured to *not* fsync() on all Append() calls
I20260811 05:16:59.973301 29297 tablet_bootstrap.cc:492] T 824263a76011420a8951bd8861f8a722 P c9c2f9de568c4ceaa17d42fac0ec64f7: No bootstrap required, opened a new log
I20260811 05:16:59.973385 29297 ts_tablet_manager.cc:1397] T 824263a76011420a8951bd8861f8a722 P c9c2f9de568c4ceaa17d42fac0ec64f7: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260811 05:16:59.973623 29303 tablet_bootstrap.cc:492] T 824263a76011420a8951bd8861f8a722 P 42cbd08113224eb086987ae22302a626: Bootstrap starting.
I20260811 05:16:59.974184 29298 raft_consensus.cc:348] T 824263a76011420a8951bd8861f8a722 P f1c5f719ec0e491b967e6bd484966f57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9c2f9de568c4ceaa17d42fac0ec64f7" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 42715 } } peers { permanent_uuid: "42cbd08113224eb086987ae22302a626" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 44925 } } peers { permanent_uuid: "f1c5f719ec0e491b967e6bd484966f57" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44515 } }
I20260811 05:16:59.974292 29303 tablet_bootstrap.cc:654] T 824263a76011420a8951bd8861f8a722 P 42cbd08113224eb086987ae22302a626: Neither blocks nor log segments found. Creating new log.
I20260811 05:16:59.974326 29298 raft_consensus.cc:374] T 824263a76011420a8951bd8861f8a722 P f1c5f719ec0e491b967e6bd484966f57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:16:59.974361 29298 raft_consensus.cc:729] T 824263a76011420a8951bd8861f8a722 P f1c5f719ec0e491b967e6bd484966f57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1c5f719ec0e491b967e6bd484966f57, State: Initialized, Role: FOLLOWER
I20260811 05:16:59.974511 29298 consensus_queue.cc:260] T 824263a76011420a8951bd8861f8a722 P f1c5f719ec0e491b967e6bd484966f57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9c2f9de568c4ceaa17d42fac0ec64f7" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 42715 } } peers { permanent_uuid: "42cbd08113224eb086987ae22302a626" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 44925 } } peers { permanent_uuid: "f1c5f719ec0e491b967e6bd484966f57" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44515 } }
I20260811 05:16:59.974596 29303 log.cc:824] T 824263a76011420a8951bd8861f8a722 P 42cbd08113224eb086987ae22302a626: Log is configured to *not* fsync() on all Append() calls
I20260811 05:16:59.974759 29298 ts_tablet_manager.cc:1428] T 824263a76011420a8951bd8861f8a722 P f1c5f719ec0e491b967e6bd484966f57: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 05:16:59.974905 29271 heartbeater.cc:503] Master 127.28.143.190:35677 was elected leader, sending a full tablet report...
I20260811 05:16:59.975431 29303 tablet_bootstrap.cc:492] T 824263a76011420a8951bd8861f8a722 P 42cbd08113224eb086987ae22302a626: No bootstrap required, opened a new log
I20260811 05:16:59.975507 29303 ts_tablet_manager.cc:1397] T 824263a76011420a8951bd8861f8a722 P 42cbd08113224eb086987ae22302a626: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 05:16:59.975811 29297 raft_consensus.cc:348] T 824263a76011420a8951bd8861f8a722 P c9c2f9de568c4ceaa17d42fac0ec64f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9c2f9de568c4ceaa17d42fac0ec64f7" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 42715 } } peers { permanent_uuid: "42cbd08113224eb086987ae22302a626" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 44925 } } peers { permanent_uuid: "f1c5f719ec0e491b967e6bd484966f57" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44515 } }
I20260811 05:16:59.976002 29297 raft_consensus.cc:374] T 824263a76011420a8951bd8861f8a722 P c9c2f9de568c4ceaa17d42fac0ec64f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:16:59.976052 29297 raft_consensus.cc:729] T 824263a76011420a8951bd8861f8a722 P c9c2f9de568c4ceaa17d42fac0ec64f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9c2f9de568c4ceaa17d42fac0ec64f7, State: Initialized, Role: FOLLOWER
I20260811 05:16:59.976199 29297 consensus_queue.cc:260] T 824263a76011420a8951bd8861f8a722 P c9c2f9de568c4ceaa17d42fac0ec64f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9c2f9de568c4ceaa17d42fac0ec64f7" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 42715 } } peers { permanent_uuid: "42cbd08113224eb086987ae22302a626" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 44925 } } peers { permanent_uuid: "f1c5f719ec0e491b967e6bd484966f57" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44515 } }
I20260811 05:16:59.976485 29297 ts_tablet_manager.cc:1428] T 824263a76011420a8951bd8861f8a722 P c9c2f9de568c4ceaa17d42fac0ec64f7: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 05:16:59.976505 28798 heartbeater.cc:503] Master 127.28.143.190:35677 was elected leader, sending a full tablet report...
I20260811 05:16:59.977865 29303 raft_consensus.cc:348] T 824263a76011420a8951bd8861f8a722 P 42cbd08113224eb086987ae22302a626 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9c2f9de568c4ceaa17d42fac0ec64f7" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 42715 } } peers { permanent_uuid: "42cbd08113224eb086987ae22302a626" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 44925 } } peers { permanent_uuid: "f1c5f719ec0e491b967e6bd484966f57" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44515 } }
I20260811 05:16:59.978012 29303 raft_consensus.cc:374] T 824263a76011420a8951bd8861f8a722 P 42cbd08113224eb086987ae22302a626 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:16:59.978035 29303 raft_consensus.cc:729] T 824263a76011420a8951bd8861f8a722 P 42cbd08113224eb086987ae22302a626 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42cbd08113224eb086987ae22302a626, State: Initialized, Role: FOLLOWER
I20260811 05:16:59.978134 29303 consensus_queue.cc:260] T 824263a76011420a8951bd8861f8a722 P 42cbd08113224eb086987ae22302a626 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9c2f9de568c4ceaa17d42fac0ec64f7" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 42715 } } peers { permanent_uuid: "42cbd08113224eb086987ae22302a626" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 44925 } } peers { permanent_uuid: "f1c5f719ec0e491b967e6bd484966f57" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44515 } }
I20260811 05:16:59.978384 29303 ts_tablet_manager.cc:1428] T 824263a76011420a8951bd8861f8a722 P 42cbd08113224eb086987ae22302a626: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 05:16:59.978436 29031 heartbeater.cc:503] Master 127.28.143.190:35677 was elected leader, sending a full tablet report...
I20260811 05:16:59.990619 29307 raft_consensus.cc:482] T 824263a76011420a8951bd8861f8a722 P 42cbd08113224eb086987ae22302a626 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:16:59.990756 29307 raft_consensus.cc:504] T 824263a76011420a8951bd8861f8a722 P 42cbd08113224eb086987ae22302a626 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9c2f9de568c4ceaa17d42fac0ec64f7" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 42715 } } peers { permanent_uuid: "42cbd08113224eb086987ae22302a626" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 44925 } } peers { permanent_uuid: "f1c5f719ec0e491b967e6bd484966f57" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44515 } }
I20260811 05:16:59.991128 29307 leader_election.cc:288] T 824263a76011420a8951bd8861f8a722 P 42cbd08113224eb086987ae22302a626 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c9c2f9de568c4ceaa17d42fac0ec64f7 (127.28.143.129:42715), f1c5f719ec0e491b967e6bd484966f57 (127.28.143.131:44515)
I20260811 05:16:59.995115 29296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:59.991399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:47027 (local address 127.28.143.129:42715)
0811 05:16:59.991469 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:59.991471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:59.991721 (+ 250us) server_negotiation.cc:407] Connection header received
0811 05:16:59.991759 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:59.991760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:59.991772 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:59.991829 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:16:59.991981 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.992683 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.994085 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.994661 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.994735 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:59.994744 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:59.995006 (+ 262us) server_negotiation.cc:299] Negotiation successful
0811 05:16:59.995053 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 05:16:59.995105 29292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:59.991398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:60969 (local address 127.28.143.131:44515)
0811 05:16:59.991455 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:59.991457 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:59.991474 (+ 17us) server_negotiation.cc:407] Connection header received
0811 05:16:59.991506 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:59.991507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:59.991516 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:59.991564 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:16:59.991760 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.992516 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.994084 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.994651 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.994735 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:59.994744 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:59.994979 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 05:16:59.995033 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 05:16:59.995075 29034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:59.991305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.131:44515 (local address 127.28.143.130:60969)
0811 05:16:59.991356 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:59.991368 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:59.991411 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:59.991588 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:59.991589 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:59.991591 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:16:59.991679 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:59.991681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.992659 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.992661 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:59.993971 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:59.993976 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.994760 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.994762 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:59.994866 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:59.994868 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 05:16:59.994935 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 05:16:59.994985 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":29}
I20260811 05:16:59.995100 29312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:59.991304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:42715 (local address 127.28.143.130:47027)
0811 05:16:59.991650 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:59.991660 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:59.991713 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:59.991834 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:59.991834 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:59.991836 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:16:59.991916 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:59.991917 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.992745 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.992746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:59.993972 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:59.993976 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:59.994768 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:59.994768 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:59.994859 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:59.994865 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 05:16:59.994984 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 05:16:59.995030 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"mutex_wait_us":182,"thread_start_us":111,"threads_started":1}
I20260811 05:16:59.995522 28715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "824263a76011420a8951bd8861f8a722" candidate_uuid: "42cbd08113224eb086987ae22302a626" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9c2f9de568c4ceaa17d42fac0ec64f7" is_pre_election: true
I20260811 05:16:59.995721 29185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "824263a76011420a8951bd8861f8a722" candidate_uuid: "42cbd08113224eb086987ae22302a626" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1c5f719ec0e491b967e6bd484966f57" is_pre_election: true
I20260811 05:16:59.995815 28715 raft_consensus.cc:2454] T 824263a76011420a8951bd8861f8a722 P c9c2f9de568c4ceaa17d42fac0ec64f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42cbd08113224eb086987ae22302a626 in term 0.
I20260811 05:16:59.995889 29185 raft_consensus.cc:2454] T 824263a76011420a8951bd8861f8a722 P f1c5f719ec0e491b967e6bd484966f57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42cbd08113224eb086987ae22302a626 in term 0.
I20260811 05:16:59.996102 28832 leader_election.cc:302] T 824263a76011420a8951bd8861f8a722 P 42cbd08113224eb086987ae22302a626 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42cbd08113224eb086987ae22302a626, c9c2f9de568c4ceaa17d42fac0ec64f7; no voters:
I20260811 05:16:59.996243 29307 raft_consensus.cc:2781] T 824263a76011420a8951bd8861f8a722 P 42cbd08113224eb086987ae22302a626 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:16:59.996305 29307 raft_consensus.cc:482] T 824263a76011420a8951bd8861f8a722 P 42cbd08113224eb086987ae22302a626 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:16:59.996338 29307 raft_consensus.cc:3037] T 824263a76011420a8951bd8861f8a722 P 42cbd08113224eb086987ae22302a626 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:16:59.997242 29307 raft_consensus.cc:504] T 824263a76011420a8951bd8861f8a722 P 42cbd08113224eb086987ae22302a626 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9c2f9de568c4ceaa17d42fac0ec64f7" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 42715 } } peers { permanent_uuid: "42cbd08113224eb086987ae22302a626" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 44925 } } peers { permanent_uuid: "f1c5f719ec0e491b967e6bd484966f57" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44515 } }
I20260811 05:16:59.997395 29307 leader_election.cc:288] T 824263a76011420a8951bd8861f8a722 P 42cbd08113224eb086987ae22302a626 [CANDIDATE]: Term 1 election: Requested vote from peers c9c2f9de568c4ceaa17d42fac0ec64f7 (127.28.143.129:42715), f1c5f719ec0e491b967e6bd484966f57 (127.28.143.131:44515)
I20260811 05:16:59.997534 28715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "824263a76011420a8951bd8861f8a722" candidate_uuid: "42cbd08113224eb086987ae22302a626" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9c2f9de568c4ceaa17d42fac0ec64f7"
I20260811 05:16:59.997581 29185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "824263a76011420a8951bd8861f8a722" candidate_uuid: "42cbd08113224eb086987ae22302a626" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1c5f719ec0e491b967e6bd484966f57"
I20260811 05:16:59.997625 28715 raft_consensus.cc:3037] T 824263a76011420a8951bd8861f8a722 P c9c2f9de568c4ceaa17d42fac0ec64f7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:16:59.997661 29185 raft_consensus.cc:3037] T 824263a76011420a8951bd8861f8a722 P f1c5f719ec0e491b967e6bd484966f57 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:16:59.998548 29185 raft_consensus.cc:2454] T 824263a76011420a8951bd8861f8a722 P f1c5f719ec0e491b967e6bd484966f57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42cbd08113224eb086987ae22302a626 in term 1.
I20260811 05:16:59.998548 28715 raft_consensus.cc:2454] T 824263a76011420a8951bd8861f8a722 P c9c2f9de568c4ceaa17d42fac0ec64f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42cbd08113224eb086987ae22302a626 in term 1.
I20260811 05:16:59.998754 28832 leader_election.cc:302] T 824263a76011420a8951bd8861f8a722 P 42cbd08113224eb086987ae22302a626 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42cbd08113224eb086987ae22302a626, c9c2f9de568c4ceaa17d42fac0ec64f7; no voters:
I20260811 05:16:59.998837 29307 raft_consensus.cc:2781] T 824263a76011420a8951bd8861f8a722 P 42cbd08113224eb086987ae22302a626 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:16:59.999017 29307 raft_consensus.cc:686] T 824263a76011420a8951bd8861f8a722 P 42cbd08113224eb086987ae22302a626 [term 1 LEADER]: Becoming Leader. State: Replica: 42cbd08113224eb086987ae22302a626, State: Running, Role: LEADER
I20260811 05:16:59.999112 29307 consensus_queue.cc:237] T 824263a76011420a8951bd8861f8a722 P 42cbd08113224eb086987ae22302a626 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9c2f9de568c4ceaa17d42fac0ec64f7" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 42715 } } peers { permanent_uuid: "42cbd08113224eb086987ae22302a626" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 44925 } } peers { permanent_uuid: "f1c5f719ec0e491b967e6bd484966f57" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44515 } }
I20260811 05:17:00.000069 28452 catalog_manager.cc:5495] T 824263a76011420a8951bd8861f8a722 P 42cbd08113224eb086987ae22302a626 reported cstate change: term changed from 0 to 1, leader changed from <none> to 42cbd08113224eb086987ae22302a626 (127.28.143.130). New cstate: current_term: 1 leader_uuid: "42cbd08113224eb086987ae22302a626" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9c2f9de568c4ceaa17d42fac0ec64f7" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 42715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42cbd08113224eb086987ae22302a626" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 44925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f1c5f719ec0e491b967e6bd484966f57" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44515 } health_report { overall_health: UNKNOWN } } }
I20260811 05:17:00.011370 29295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:00.009047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51492 (local address 127.28.143.130:44925)
0811 05:17:00.009112 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:00.009114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:00.009134 (+ 20us) server_negotiation.cc:407] Connection header received
0811 05:17:00.009186 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:00.009187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:00.009206 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:00.009270 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:17:00.009388 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:00.009871 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:00.010622 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:00.010814 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:00.010871 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:00.011034 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:17:00.011146 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:17:00.011221 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:00.011241 (+ 20us) server_negotiation.cc:299] Negotiation successful
0811 05:17:00.011263 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 05:17:00.013406 29185 raft_consensus.cc:1261] T 824263a76011420a8951bd8861f8a722 P f1c5f719ec0e491b967e6bd484966f57 [term 1 FOLLOWER]: Refusing update from remote peer 42cbd08113224eb086987ae22302a626: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:17:00.013419 28715 raft_consensus.cc:1261] T 824263a76011420a8951bd8861f8a722 P c9c2f9de568c4ceaa17d42fac0ec64f7 [term 1 FOLLOWER]: Refusing update from remote peer 42cbd08113224eb086987ae22302a626: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:17:00.013784 29313 consensus_queue.cc:1035] T 824263a76011420a8951bd8861f8a722 P 42cbd08113224eb086987ae22302a626 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1c5f719ec0e491b967e6bd484966f57" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 44515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:17:00.013933 29307 consensus_queue.cc:1035] T 824263a76011420a8951bd8861f8a722 P 42cbd08113224eb086987ae22302a626 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9c2f9de568c4ceaa17d42fac0ec64f7" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 42715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:17:00.016495 29317 mvcc.cc:205] Tried to move back new op lower bound from 7317198520372101120 to 7317198520317014016. Current Snapshot: MvccSnapshot[applied={T|T < 7317198520372101120}]
I20260811 05:17:00.016602 29320 mvcc.cc:205] Tried to move back new op lower bound from 7317198520372101120 to 7317198520317014016. Current Snapshot: MvccSnapshot[applied={T|T < 7317198520372101120}]
I20260811 05:17:00.019397 28451 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49452:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:17:00.019515 28451 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49452:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:17:00.020308 28451 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 40c182c04ee447e6919cad34ff78baca: Sending DeleteTablet for 3 replicas of tablet 824263a76011420a8951bd8861f8a722
I20260811 05:17:00.020764 29143 tablet_service.cc:1514] Processing DeleteTablet for tablet 824263a76011420a8951bd8861f8a722 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:17:00 UTC) from {username='slave'} at 127.0.0.1:35116
I20260811 05:17:00.020794 28913 tablet_service.cc:1514] Processing DeleteTablet for tablet 824263a76011420a8951bd8861f8a722 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:17:00 UTC) from {username='slave'} at 127.0.0.1:51490
I20260811 05:17:00.021060 28687 tablet_service.cc:1514] Processing DeleteTablet for tablet 824263a76011420a8951bd8861f8a722 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:17:00 UTC) from {username='slave'} at 127.0.0.1:35990
I20260811 05:17:00.021349 29336 tablet_replica.cc:330] T 824263a76011420a8951bd8861f8a722 P c9c2f9de568c4ceaa17d42fac0ec64f7: stopping tablet replica
I20260811 05:17:00.021347 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 28573
I20260811 05:17:00.032845 29332 tablet_replica.cc:330] T 824263a76011420a8951bd8861f8a722 P f1c5f719ec0e491b967e6bd484966f57: stopping tablet replica
I20260811 05:17:00.033038 29332 raft_consensus.cc:2229] T 824263a76011420a8951bd8861f8a722 P f1c5f719ec0e491b967e6bd484966f57 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:17:00.033183 29332 raft_consensus.cc:2258] T 824263a76011420a8951bd8861f8a722 P f1c5f719ec0e491b967e6bd484966f57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:17:00.033341 29331 tablet_replica.cc:330] T 824263a76011420a8951bd8861f8a722 P 42cbd08113224eb086987ae22302a626: stopping tablet replica
I20260811 05:17:00.033455 29331 raft_consensus.cc:2229] T 824263a76011420a8951bd8861f8a722 P 42cbd08113224eb086987ae22302a626 [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:17:00.033587 29331 raft_consensus.cc:2258] T 824263a76011420a8951bd8861f8a722 P 42cbd08113224eb086987ae22302a626 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:17:00.034371 29331 ts_tablet_manager.cc:1905] T 824263a76011420a8951bd8861f8a722 P 42cbd08113224eb086987ae22302a626: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:17:00.036509 29331 ts_tablet_manager.cc:1918] T 824263a76011420a8951bd8861f8a722 P 42cbd08113224eb086987ae22302a626: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:17:00.036607 29331 log.cc:1196] T 824263a76011420a8951bd8861f8a722 P 42cbd08113224eb086987ae22302a626: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-1/wal/wals/824263a76011420a8951bd8861f8a722
I20260811 05:17:00.036948 29331 ts_tablet_manager.cc:1939] T 824263a76011420a8951bd8861f8a722 P 42cbd08113224eb086987ae22302a626: Deleting consensus metadata
I20260811 05:17:00.037462 28430 catalog_manager.cc:4841] TS 42cbd08113224eb086987ae22302a626 (127.28.143.130:44925): tablet 824263a76011420a8951bd8861f8a722 (table test-table [id=9413220d20c44d6995b0663d8be7721d]) successfully deleted
W20260811 05:17:00.039119 28433 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.143.129:42715 (error 108)
I20260811 05:17:00.039238 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 28804
I20260811 05:17:00.043516 29332 ts_tablet_manager.cc:1905] T 824263a76011420a8951bd8861f8a722 P f1c5f719ec0e491b967e6bd484966f57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:17:00.045516 29290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:00.045211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:42715 (local address 127.0.0.1:36004)
0811 05:17:00.045293 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:00.045369 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.129:42715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 05:17:00.045763 28433 catalog_manager.cc:4584] TS c9c2f9de568c4ceaa17d42fac0ec64f7 (127.28.143.129:42715): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 824263a76011420a8951bd8861f8a722: Network error: Client connection negotiation failed: client connection to 127.28.143.129:42715: connect: Connection refused (error 111)
I20260811 05:17:00.046162 29332 ts_tablet_manager.cc:1918] T 824263a76011420a8951bd8861f8a722 P f1c5f719ec0e491b967e6bd484966f57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:17:00.046260 29332 log.cc:1196] T 824263a76011420a8951bd8861f8a722 P f1c5f719ec0e491b967e6bd484966f57: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425159060450-29246-0/minicluster-data/ts-2/wal/wals/824263a76011420a8951bd8861f8a722
I20260811 05:17:00.046533 29332 ts_tablet_manager.cc:1939] T 824263a76011420a8951bd8861f8a722 P f1c5f719ec0e491b967e6bd484966f57: Deleting consensus metadata
I20260811 05:17:00.047072 28429 catalog_manager.cc:4841] TS f1c5f719ec0e491b967e6bd484966f57 (127.28.143.131:44515): tablet 824263a76011420a8951bd8861f8a722 (table test-table [id=9413220d20c44d6995b0663d8be7721d]) successfully deleted
I20260811 05:17:00.052536 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 29035
I20260811 05:17:00.074157 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 28404
2026-08-11T05:17:00Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (747 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:17:00 dist-test-slave-f7th krb5kdc[29341](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:17:00 dist-test-slave-f7th krb5kdc[29341](info): set up 2 sockets
Aug 11 05:17:00 dist-test-slave-f7th krb5kdc[29341](info): commencing operation
krb5kdc: starting...
W20260811 05:17:01.287163 29246 mini_kdc.cc:121] Time spent starting KDC: real 1.189s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Aug 11 05:17:01 dist-test-slave-f7th krb5kdc[29341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:17:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:17:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
I20260811 05:17:01.428239 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:38797
--webserver_interface=127.28.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:34065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:38797
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:01.516659 29360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:01.516904 29360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:01.516928 29360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:01.518178 29360 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:01.518232 29360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:01.518244 29360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:01.518256 29360 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:01.518267 29360 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:01.519888 29360 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:34065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:38797
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:38797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:17:01.520169 29360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:01.520459 29360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 05:17:01.525789 29360 server_base.cc:1034] running on GCE node
W20260811 05:17:01.525713 29371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:01.525751 29367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:01.525713 29373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:01.526352 29360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:01.526654 29360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:01.527824 29360 hybrid_clock.cc:648] HybridClock initialized: now 1786425421527782 us; error 61 us; skew 500 ppm
Aug 11 05:17:01 dist-test-slave-f7th krb5kdc[29341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425421, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:01.530759 29360 init.cc:377] Logged in from keytab as kudu/127.28.143.190@KRBTEST.COM (short username kudu)
I20260811 05:17:01.531320 29360 webserver.cc:460] Webserver started at http://127.28.143.190:33111/ using document root <none> and password file <none>
I20260811 05:17:01.531550 29360 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:01.531603 29360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:01.531744 29360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:01.532976 29360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "1c8fb3a494ff49b294e14435bc7f970e"
format_stamp: "Formatted at 2026-08-11 05:17:01 on dist-test-slave-f7th"
server_key: "409661457706161f0120ac135666a924"
server_key_iv: "c1b5cb86a8603f0db1cf978a7581d484"
server_key_version: "encryptionkey@0"
I20260811 05:17:01.533334 29360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "1c8fb3a494ff49b294e14435bc7f970e"
format_stamp: "Formatted at 2026-08-11 05:17:01 on dist-test-slave-f7th"
server_key: "409661457706161f0120ac135666a924"
server_key_iv: "c1b5cb86a8603f0db1cf978a7581d484"
server_key_version: "encryptionkey@0"
I20260811 05:17:01.534857 29360 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 05:17:01.535818 29389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:01.536156 29360 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:17:01.536273 29360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "1c8fb3a494ff49b294e14435bc7f970e"
format_stamp: "Formatted at 2026-08-11 05:17:01 on dist-test-slave-f7th"
server_key: "409661457706161f0120ac135666a924"
server_key_iv: "c1b5cb86a8603f0db1cf978a7581d484"
server_key_version: "encryptionkey@0"
I20260811 05:17:01.536347 29360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:01.547375 29360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:01.549713 29360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:01.549873 29360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:01.553997 29360 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:38797
I20260811 05:17:01.554049 29486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:38797 every 8 connection(s)
I20260811 05:17:01.554396 29360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:17:01.554875 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 29360
I20260811 05:17:01.554984 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:17:01.555173 29246 external_mini_cluster.cc:1428] Setting key 6abc4b6f5d2c3c352b0a86397c4c830e
I20260811 05:17:01.555334 29488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:01.558698 29488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c8fb3a494ff49b294e14435bc7f970e: Bootstrap starting.
I20260811 05:17:01.559435 29488 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c8fb3a494ff49b294e14435bc7f970e: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:01.559794 29488 log.cc:824] T 00000000000000000000000000000000 P 1c8fb3a494ff49b294e14435bc7f970e: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:01.560518 29488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c8fb3a494ff49b294e14435bc7f970e: No bootstrap required, opened a new log
Aug 11 05:17:01 dist-test-slave-f7th krb5kdc[29341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:17:01.562819 29488 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c8fb3a494ff49b294e14435bc7f970e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c8fb3a494ff49b294e14435bc7f970e" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 38797 } }
I20260811 05:17:01.563004 29488 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c8fb3a494ff49b294e14435bc7f970e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:01.563066 29488 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c8fb3a494ff49b294e14435bc7f970e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c8fb3a494ff49b294e14435bc7f970e, State: Initialized, Role: FOLLOWER
I20260811 05:17:01.563213 29488 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c8fb3a494ff49b294e14435bc7f970e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c8fb3a494ff49b294e14435bc7f970e" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 38797 } }
I20260811 05:17:01.563289 29488 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c8fb3a494ff49b294e14435bc7f970e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:17:01.563313 29488 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c8fb3a494ff49b294e14435bc7f970e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:17:01.563359 29488 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c8fb3a494ff49b294e14435bc7f970e [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:01.563903 29488 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c8fb3a494ff49b294e14435bc7f970e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c8fb3a494ff49b294e14435bc7f970e" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 38797 } }
I20260811 05:17:01.564000 29488 leader_election.cc:302] T 00000000000000000000000000000000 P 1c8fb3a494ff49b294e14435bc7f970e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c8fb3a494ff49b294e14435bc7f970e; no voters:
I20260811 05:17:01.564175 29488 leader_election.cc:288] T 00000000000000000000000000000000 P 1c8fb3a494ff49b294e14435bc7f970e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:17:01.564234 29493 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c8fb3a494ff49b294e14435bc7f970e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:01.564437 29488 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c8fb3a494ff49b294e14435bc7f970e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:01.564445 29493 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c8fb3a494ff49b294e14435bc7f970e [term 1 LEADER]: Becoming Leader. State: Replica: 1c8fb3a494ff49b294e14435bc7f970e, State: Running, Role: LEADER
I20260811 05:17:01.564679 29493 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c8fb3a494ff49b294e14435bc7f970e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c8fb3a494ff49b294e14435bc7f970e" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 38797 } }
I20260811 05:17:01.565042 29496 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c8fb3a494ff49b294e14435bc7f970e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c8fb3a494ff49b294e14435bc7f970e. Latest consensus state: current_term: 1 leader_uuid: "1c8fb3a494ff49b294e14435bc7f970e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c8fb3a494ff49b294e14435bc7f970e" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 38797 } } }
I20260811 05:17:01.565061 29494 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c8fb3a494ff49b294e14435bc7f970e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c8fb3a494ff49b294e14435bc7f970e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c8fb3a494ff49b294e14435bc7f970e" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 38797 } } }
I20260811 05:17:01.565169 29496 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c8fb3a494ff49b294e14435bc7f970e [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:01.565227 29494 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c8fb3a494ff49b294e14435bc7f970e [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:01.565433 29505 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:17:01.566172 29505 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:17:01.566560 29491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:01.555702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57362 (local address 127.28.143.190:38797)
0811 05:17:01.555871 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:01.555879 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:01.555905 (+ 26us) server_negotiation.cc:407] Connection header received
0811 05:17:01.556524 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:01.556532 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:01.556583 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:01.556739 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:01.557532 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:01.558431 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:01.559430 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:01.559895 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:01.559987 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:01.562601 (+ 2614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:01.562606 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:01.562606 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:01.562619 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:01.564508 (+ 1889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:01.564906 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:01.564909 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:01.564910 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:01.565021 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:01.565258 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:01.565258 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:01.565259 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:01.565596 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:01.566123 (+ 527us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:01.566200 (+ 77us) server_negotiation.cc:299] Negotiation successful
0811 05:17:01.566350 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":73,"threads_started":1}
I20260811 05:17:01.567792 29518 mvcc.cc:205] Tried to move back new op lower bound from 7317198526739144704 to 7317198526729236480. Current Snapshot: MvccSnapshot[applied={T|T < 7317198526739144704}]
I20260811 05:17:01.568171 29505 catalog_manager.cc:1347] Generated new cluster ID: c1c4bd8e7b864658997de13c4208ae8e
I20260811 05:17:01.568224 29505 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:17:01.578104 29505 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:17:01.578882 29505 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:17:01.593859 29505 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c8fb3a494ff49b294e14435bc7f970e: Generated new TSK 0
I20260811 05:17:01.594105 29505 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.143.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab.
Entry for principal kudu/127.28.143.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab.
I20260811 05:17:01.641800 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.129:0
--local_ip_for_outbound_sockets=127.28.143.129
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:38797
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:34065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:01.732199 29524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:01.732422 29524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:01.732457 29524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:01.733722 29524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:01.733773 29524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:01.733811 29524 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.129
I20260811 05:17:01.735337 29524 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:34065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:38797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.129
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:17:01.735601 29524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:01.735836 29524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:01.741207 29532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:01.741223 29535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:01.741189 29531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:01.741250 29524 server_base.cc:1034] running on GCE node
I20260811 05:17:01.741616 29524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:01.741873 29524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:01.743067 29524 hybrid_clock.cc:648] HybridClock initialized: now 1786425421743048 us; error 27 us; skew 500 ppm
Aug 11 05:17:01 dist-test-slave-f7th krb5kdc[29341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425421, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:01.746518 29524 init.cc:377] Logged in from keytab as kudu/127.28.143.129@KRBTEST.COM (short username kudu)
I20260811 05:17:01.747140 29524 webserver.cc:460] Webserver started at http://127.28.143.129:39203/ using document root <none> and password file <none>
I20260811 05:17:01.747354 29524 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:01.747398 29524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:01.747495 29524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:01.748596 29524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-0/data/instance:
uuid: "4f4fb338291042f3a0c8fb4de21bbf29"
format_stamp: "Formatted at 2026-08-11 05:17:01 on dist-test-slave-f7th"
server_key: "ea3e6a072d111e5ee4f96424bc8ea475"
server_key_iv: "b822afc3a687361b84ed92fce892ac06"
server_key_version: "encryptionkey@0"
I20260811 05:17:01.749001 29524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance:
uuid: "4f4fb338291042f3a0c8fb4de21bbf29"
format_stamp: "Formatted at 2026-08-11 05:17:01 on dist-test-slave-f7th"
server_key: "ea3e6a072d111e5ee4f96424bc8ea475"
server_key_iv: "b822afc3a687361b84ed92fce892ac06"
server_key_version: "encryptionkey@0"
I20260811 05:17:01.750463 29524 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 05:17:01.751315 29545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:01.751518 29524 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 05:17:01.751624 29524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-0/wal
uuid: "4f4fb338291042f3a0c8fb4de21bbf29"
format_stamp: "Formatted at 2026-08-11 05:17:01 on dist-test-slave-f7th"
server_key: "ea3e6a072d111e5ee4f96424bc8ea475"
server_key_iv: "b822afc3a687361b84ed92fce892ac06"
server_key_version: "encryptionkey@0"
I20260811 05:17:01.751698 29524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:01.764308 29524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:01.767127 29524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:01.767297 29524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:01.767588 29524 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:17:01.767941 29524 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:17:01.767979 29524 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:01.768009 29524 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:17:01.768026 29524 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:01.774698 29524 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.129:36407
I20260811 05:17:01.774746 29730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.129:36407 every 8 connection(s)
I20260811 05:17:01.775104 29524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
I20260811 05:17:01.779129 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 29524
I20260811 05:17:01.779249 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance
I20260811 05:17:01.779456 29246 external_mini_cluster.cc:1428] Setting key c014402d073b3474ced34e0e96a48e5f
Aug 11 05:17:01 dist-test-slave-f7th krb5kdc[29341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425421, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.28.143.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.130@KRBTEST.COM" created.
I20260811 05:17:01.791663 29491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:01.775886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:42633 (local address 127.28.143.190:38797)
0811 05:17:01.776007 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:01.776009 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:01.776154 (+ 145us) server_negotiation.cc:407] Connection header received
0811 05:17:01.776717 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:01.776719 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:01.776732 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:01.776780 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:01.777845 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:01.778360 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:01.779594 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:01.779818 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:01.779903 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:01.782274 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:01.782276 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:01.782276 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:01.782283 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:01.789965 (+ 7682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:01.790528 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:01.790529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:01.790530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:01.790593 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:01.790878 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:01.790879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:01.790879 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:01.791010 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:01.791081 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:01.791434 (+ 353us) server_negotiation.cc:299] Negotiation successful
0811 05:17:01.791552 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 05:17:01.791839 29733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:01.775832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:38797 (local address 127.28.143.129:42633)
0811 05:17:01.776022 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:01.776070 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:01.776578 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:01.776854 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:01.776858 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:01.777526 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:01.777737 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:01.777742 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:01.778459 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:01.778460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:01.779494 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:01.779497 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:01.779935 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:01.779936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:01.780065 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:01.780689 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:17:01.780699 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:01.782163 (+ 1464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:01.790089 (+ 7926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:01.790108 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:01.790112 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:01.790441 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:01.790656 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:01.790657 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:01.790657 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:01.790814 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:01.791106 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:01.791117 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:01.791290 (+ 173us) client_negotiation.cc:769] Sending connection context
0811 05:17:01.791400 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 05:17:01.791590 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":87,"threads_started":1}
I20260811 05:17:01.792431 29731 heartbeater.cc:347] Connected to a master server at 127.28.143.190:38797
I20260811 05:17:01.792551 29731 heartbeater.cc:464] Registering TS with master...
I20260811 05:17:01.792801 29731 heartbeater.cc:511] Master 127.28.143.190:38797 requested a full tablet report, sending...
I20260811 05:17:01.793380 29422 ts_manager.cc:194] Registered new tserver with Master: 4f4fb338291042f3a0c8fb4de21bbf29 (127.28.143.129:36407)
I20260811 05:17:01.794108 29422 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.129@KRBTEST.COM'} at 127.28.143.129:42633
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab.
Entry for principal kudu/127.28.143.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab.
I20260811 05:17:01.808347 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.130:0
--local_ip_for_outbound_sockets=127.28.143.130
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:38797
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:34065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:01.893097 29738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:01.893347 29738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:01.893374 29738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:01.894786 29738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:01.894848 29738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:01.894879 29738 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.130
I20260811 05:17:01.896672 29738 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:34065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:38797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.130
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:17:01.896984 29738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:01.897243 29738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:01.902571 29745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:01.902614 29744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:01.902539 29748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:01.902984 29738 server_base.cc:1034] running on GCE node
I20260811 05:17:01.903234 29738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:01.903471 29738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:01.904693 29738 hybrid_clock.cc:648] HybridClock initialized: now 1786425421904664 us; error 35 us; skew 500 ppm
Aug 11 05:17:01 dist-test-slave-f7th krb5kdc[29341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425421, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:01.907630 29738 init.cc:377] Logged in from keytab as kudu/127.28.143.130@KRBTEST.COM (short username kudu)
I20260811 05:17:01.908149 29738 webserver.cc:460] Webserver started at http://127.28.143.130:45581/ using document root <none> and password file <none>
I20260811 05:17:01.908385 29738 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:01.908443 29738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:01.908573 29738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:01.909662 29738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-1/data/instance:
uuid: "b405c1e0702e4a3786c9610471281200"
format_stamp: "Formatted at 2026-08-11 05:17:01 on dist-test-slave-f7th"
server_key: "228a17fdc1306594b5fe918e0ba8bc85"
server_key_iv: "7f9d055267a0c71d6cddbadb7176b96c"
server_key_version: "encryptionkey@0"
I20260811 05:17:01.910015 29738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance:
uuid: "b405c1e0702e4a3786c9610471281200"
format_stamp: "Formatted at 2026-08-11 05:17:01 on dist-test-slave-f7th"
server_key: "228a17fdc1306594b5fe918e0ba8bc85"
server_key_iv: "7f9d055267a0c71d6cddbadb7176b96c"
server_key_version: "encryptionkey@0"
I20260811 05:17:01.911369 29738 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 05:17:01.912211 29759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:01.912760 29738 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:17:01.912864 29738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-1/wal
uuid: "b405c1e0702e4a3786c9610471281200"
format_stamp: "Formatted at 2026-08-11 05:17:01 on dist-test-slave-f7th"
server_key: "228a17fdc1306594b5fe918e0ba8bc85"
server_key_iv: "7f9d055267a0c71d6cddbadb7176b96c"
server_key_version: "encryptionkey@0"
I20260811 05:17:01.912947 29738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:01.923254 29738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:01.925709 29738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:01.925858 29738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:01.926103 29738 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:17:01.926426 29738 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:17:01.926462 29738 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:01.926493 29738 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:17:01.926524 29738 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:01.932319 29738 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.130:36323
I20260811 05:17:01.932363 29961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.130:36323 every 8 connection(s)
I20260811 05:17:01.932744 29738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
I20260811 05:17:01.934585 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 29738
I20260811 05:17:01.934677 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance
I20260811 05:17:01.934854 29246 external_mini_cluster.cc:1428] Setting key 08a03dd7eb1a4fbe9fd4bba4218296af
Aug 11 05:17:01 dist-test-slave-f7th krb5kdc[29341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425421, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:17:01.943259 29491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:01.933531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:42803 (local address 127.28.143.190:38797)
0811 05:17:01.933592 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:01.933594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:01.933779 (+ 185us) server_negotiation.cc:407] Connection header received
0811 05:17:01.934328 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:01.934330 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:01.934384 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:01.934450 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:01.935663 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:01.936305 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:01.937415 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:01.937652 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:01.937715 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:01.940347 (+ 2632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:01.940348 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:01.940349 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:01.940358 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:01.941769 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:01.942298 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:01.942299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:01.942300 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:01.942349 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:01.942628 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:01.942629 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:01.942629 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:01.942737 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:01.942783 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:01.943121 (+ 338us) server_negotiation.cc:299] Negotiation successful
0811 05:17:01.943190 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 05:17:01.943611 29967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:01.933506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:38797 (local address 127.28.143.130:42803)
0811 05:17:01.933657 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:01.933695 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:01.934218 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:01.934600 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:01.934604 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:01.935300 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:01.935567 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:01.935574 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:01.936446 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:01.936447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:01.937316 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:01.937319 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:01.937800 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:01.937801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:01.937929 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:01.938578 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:17:01.938587 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:01.940208 (+ 1621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:01.941873 (+ 1665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:01.941884 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:01.941887 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:01.942217 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:01.942407 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:01.942408 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:01.942408 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:01.942566 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:01.942794 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:01.942800 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:01.942987 (+ 187us) client_negotiation.cc:769] Sending connection context
0811 05:17:01.943113 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 05:17:01.943292 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":74,"threads_started":1}
I20260811 05:17:01.944334 29963 heartbeater.cc:347] Connected to a master server at 127.28.143.190:38797
WARNING: no policy specified for kudu/127.28.143.131@KRBTEST.COM; defaulting to no policy
I20260811 05:17:01.944470 29963 heartbeater.cc:464] Registering TS with master...
I20260811 05:17:01.944723 29963 heartbeater.cc:511] Master 127.28.143.190:38797 requested a full tablet report, sending...
I20260811 05:17:01.945201 29422 ts_manager.cc:194] Registered new tserver with Master: b405c1e0702e4a3786c9610471281200 (127.28.143.130:36323)
I20260811 05:17:01.945595 29422 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.130@KRBTEST.COM'} at 127.28.143.130:42803
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab.
Entry for principal kudu/127.28.143.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab.
I20260811 05:17:01.968580 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.131:0
--local_ip_for_outbound_sockets=127.28.143.131
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:38797
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:34065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:02.056749 29972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:02.056975 29972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:02.056999 29972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:02.058179 29972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:02.058229 29972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:02.058255 29972 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.131
I20260811 05:17:02.059782 29972 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:34065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:38797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.131
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:17:02.060034 29972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:02.060264 29972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:02.065344 29982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:02.065371 29984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:02.065366 29981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:02.065752 29972 server_base.cc:1034] running on GCE node
I20260811 05:17:02.065985 29972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:02.066300 29972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:02.067649 29972 hybrid_clock.cc:648] HybridClock initialized: now 1786425422067634 us; error 34 us; skew 500 ppm
Aug 11 05:17:02 dist-test-slave-f7th krb5kdc[29341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425422, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:02.070232 29972 init.cc:377] Logged in from keytab as kudu/127.28.143.131@KRBTEST.COM (short username kudu)
I20260811 05:17:02.070730 29972 webserver.cc:460] Webserver started at http://127.28.143.131:35195/ using document root <none> and password file <none>
I20260811 05:17:02.070957 29972 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:02.071007 29972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:02.071153 29972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:02.072239 29972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-2/data/instance:
uuid: "cf2fa555cbfa41e38d09bd1a962ced88"
format_stamp: "Formatted at 2026-08-11 05:17:02 on dist-test-slave-f7th"
server_key: "a7560a94adb6672a084ff05ef42587d7"
server_key_iv: "2473d98ccce2d9032ec21358d5658771"
server_key_version: "encryptionkey@0"
I20260811 05:17:02.072593 29972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance:
uuid: "cf2fa555cbfa41e38d09bd1a962ced88"
format_stamp: "Formatted at 2026-08-11 05:17:02 on dist-test-slave-f7th"
server_key: "a7560a94adb6672a084ff05ef42587d7"
server_key_iv: "2473d98ccce2d9032ec21358d5658771"
server_key_version: "encryptionkey@0"
I20260811 05:17:02.073993 29972 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:17:02.074810 29999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:02.074990 29972 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:17:02.075096 29972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-2/wal
uuid: "cf2fa555cbfa41e38d09bd1a962ced88"
format_stamp: "Formatted at 2026-08-11 05:17:02 on dist-test-slave-f7th"
server_key: "a7560a94adb6672a084ff05ef42587d7"
server_key_iv: "2473d98ccce2d9032ec21358d5658771"
server_key_version: "encryptionkey@0"
I20260811 05:17:02.075158 29972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:02.100283 29972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:02.102674 29972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:02.102806 29972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:02.103035 29972 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:17:02.103374 29972 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:17:02.103420 29972 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:02.103447 29972 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:17:02.103458 29972 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:02.109138 29972 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.131:36883
I20260811 05:17:02.109191 30205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.131:36883 every 8 connection(s)
I20260811 05:17:02.109508 29972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
Aug 11 05:17:02 dist-test-slave-f7th krb5kdc[29341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425422, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:17:02.115650 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 29972
I20260811 05:17:02.115753 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance
I20260811 05:17:02.115913 29246 external_mini_cluster.cc:1428] Setting key 8d7c20be879c4d002265da74de0fadfd
I20260811 05:17:02.118748 29491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:02.110234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:40425 (local address 127.28.143.190:38797)
0811 05:17:02.110293 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:02.110295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:02.110500 (+ 205us) server_negotiation.cc:407] Connection header received
0811 05:17:02.111024 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:02.111025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:02.111037 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:02.111089 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:02.112097 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:02.112595 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:02.113593 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:02.113784 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:02.113840 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:02.116137 (+ 2297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:02.116139 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:02.116139 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:02.116149 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:02.117652 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:02.118035 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:02.118036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:02.118036 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:02.118087 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:02.118254 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:02.118255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:02.118255 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:02.118353 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:02.118398 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:02.118614 (+ 216us) server_negotiation.cc:299] Negotiation successful
0811 05:17:02.118684 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 05:17:02.118918 30211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:02.110183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:38797 (local address 127.28.143.131:40425)
0811 05:17:02.110369 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:02.110409 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:02.110908 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:02.111160 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:02.111165 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:02.111788 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:02.111993 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:02.111998 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:02.112689 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:02.112690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:02.113503 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:02.113506 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:02.113855 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:02.113856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:02.113985 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:02.114488 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:17:02.114497 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:02.116035 (+ 1538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:02.117722 (+ 1687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:02.117728 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:02.117730 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:02.117976 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:02.118127 (+ 151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:02.118128 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:02.118128 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:02.118217 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:02.118395 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:02.118397 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:02.118519 (+ 122us) client_negotiation.cc:769] Sending connection context
0811 05:17:02.118599 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 05:17:02.118721 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":89,"threads_started":1}
I20260811 05:17:02.119361 30207 heartbeater.cc:347] Connected to a master server at 127.28.143.190:38797
I20260811 05:17:02.119458 30207 heartbeater.cc:464] Registering TS with master...
I20260811 05:17:02.119635 30207 heartbeater.cc:511] Master 127.28.143.190:38797 requested a full tablet report, sending...
I20260811 05:17:02.120077 29422 ts_manager.cc:194] Registered new tserver with Master: cf2fa555cbfa41e38d09bd1a962ced88 (127.28.143.131:36883)
I20260811 05:17:02.120465 29422 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.131@KRBTEST.COM'} at 127.28.143.131:40425
I20260811 05:17:02.127805 29246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:17:02.135329 29491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:02.129201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57364 (local address 127.28.143.190:38797)
0811 05:17:02.129249 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:02.129251 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:02.129352 (+ 101us) server_negotiation.cc:407] Connection header received
0811 05:17:02.129398 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:02.129399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:02.129408 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:02.129458 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:02.130032 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:02.130529 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:02.131419 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:02.131611 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:02.131674 (+ 63us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:02.133049 (+ 1375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:02.133050 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:02.133051 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:02.133058 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:02.134309 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:02.134699 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:02.134700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:02.134700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:02.134751 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:02.134904 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:02.134904 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:02.134904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:02.135051 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:02.135130 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:02.135212 (+ 82us) server_negotiation.cc:299] Negotiation successful
0811 05:17:02.135262 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 05:17:02.136902 29422 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57364:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:17:02.138675 29422 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:17:02.145556 30223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:02.141117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.131:36883 (local address 127.0.0.1:56634)
0811 05:17:02.141271 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:02.141311 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:02.141376 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:02.141948 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:02.141949 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:02.141951 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:02.141983 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:02.141984 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:02.143254 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:02.143255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:02.144340 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:02.144342 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:02.145229 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:02.145231 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:02.145324 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:02.145339 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 05:17:02.145410 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 05:17:02.145469 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":85,"thread_start_us":45,"threads_started":1}
I20260811 05:17:02.145576 30233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:02.141628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56634 (local address 127.28.143.131:36883)
0811 05:17:02.141808 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:02.141811 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:02.141822 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:17:02.141857 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:02.141859 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:02.141875 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:02.141961 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:02.142035 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:02.143150 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:02.144416 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:02.145128 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:02.145210 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:02.145269 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:02.145439 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 05:17:02.145490 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":84,"threads_started":1}
I20260811 05:17:02.145655 30227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:02.141119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:36323 (local address 127.0.0.1:35522)
0811 05:17:02.141798 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:02.141809 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:02.141900 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:02.142285 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:02.142285 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:02.142287 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:02.142324 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:02.142325 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:02.143157 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:02.143158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:02.144560 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:02.144562 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:02.145435 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:02.145436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:02.145490 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:02.145494 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:17:02.145544 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 05:17:02.145579 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"mutex_wait_us":61,"thread_start_us":112,"threads_started":1}
I20260811 05:17:02.145797 30232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:02.141452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35522 (local address 127.28.143.130:36323)
0811 05:17:02.141630 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:02.141633 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:02.141872 (+ 239us) server_negotiation.cc:407] Connection header received
0811 05:17:02.142122 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:02.142124 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:02.142140 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:02.142210 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:02.142389 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:02.143057 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:02.144659 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:02.145354 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:02.145418 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:02.145469 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:02.145554 (+ 85us) server_negotiation.cc:299] Negotiation successful
0811 05:17:02.145596 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":81,"threads_started":1}
I20260811 05:17:02.145906 30228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:02.141117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:36407 (local address 127.0.0.1:51094)
0811 05:17:02.141273 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:02.141311 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:02.141376 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:02.141750 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:02.141753 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:02.141756 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:02.141831 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:02.141833 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:02.143336 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:02.143336 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:02.144488 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:02.144491 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:02.145680 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:02.145681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:02.145741 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:02.145745 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:17:02.145798 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 05:17:02.145835 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":29,"thread_start_us":86,"threads_started":1}
I20260811 05:17:02.145952 30229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:02.141393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51094 (local address 127.28.143.129:36407)
0811 05:17:02.141560 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:02.141565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:02.141576 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:17:02.141616 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:02.141618 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:02.141636 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:02.141712 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:02.141994 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:02.143147 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:02.144586 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:02.145136 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:02.145213 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:02.145270 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:02.145829 (+ 559us) server_negotiation.cc:299] Negotiation successful
0811 05:17:02.145875 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":86,"threads_started":1}
I20260811 05:17:02.146787 29626 tablet_service.cc:1467] Processing CreateTablet for tablet bf679b06ac034ca09579ad78fe5532dc (DEFAULT_TABLE table=test-table [id=903db0e422be487ba87b2be4bef67e66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:17:02.146751 29849 tablet_service.cc:1467] Processing CreateTablet for tablet bf679b06ac034ca09579ad78fe5532dc (DEFAULT_TABLE table=test-table [id=903db0e422be487ba87b2be4bef67e66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:17:02.146881 30085 tablet_service.cc:1467] Processing CreateTablet for tablet bf679b06ac034ca09579ad78fe5532dc (DEFAULT_TABLE table=test-table [id=903db0e422be487ba87b2be4bef67e66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:17:02.147158 29626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf679b06ac034ca09579ad78fe5532dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:02.147159 29849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf679b06ac034ca09579ad78fe5532dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:02.147171 30085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf679b06ac034ca09579ad78fe5532dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:02.149847 30237 tablet_bootstrap.cc:492] T bf679b06ac034ca09579ad78fe5532dc P 4f4fb338291042f3a0c8fb4de21bbf29: Bootstrap starting.
I20260811 05:17:02.150475 30238 tablet_bootstrap.cc:492] T bf679b06ac034ca09579ad78fe5532dc P b405c1e0702e4a3786c9610471281200: Bootstrap starting.
I20260811 05:17:02.150825 30237 tablet_bootstrap.cc:654] T bf679b06ac034ca09579ad78fe5532dc P 4f4fb338291042f3a0c8fb4de21bbf29: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:02.151115 30238 tablet_bootstrap.cc:654] T bf679b06ac034ca09579ad78fe5532dc P b405c1e0702e4a3786c9610471281200: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:02.151438 30237 log.cc:824] T bf679b06ac034ca09579ad78fe5532dc P 4f4fb338291042f3a0c8fb4de21bbf29: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:02.151713 30238 log.cc:824] T bf679b06ac034ca09579ad78fe5532dc P b405c1e0702e4a3786c9610471281200: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:02.152056 30239 tablet_bootstrap.cc:492] T bf679b06ac034ca09579ad78fe5532dc P cf2fa555cbfa41e38d09bd1a962ced88: Bootstrap starting.
I20260811 05:17:02.152334 30237 tablet_bootstrap.cc:492] T bf679b06ac034ca09579ad78fe5532dc P 4f4fb338291042f3a0c8fb4de21bbf29: No bootstrap required, opened a new log
I20260811 05:17:02.152402 30237 ts_tablet_manager.cc:1397] T bf679b06ac034ca09579ad78fe5532dc P 4f4fb338291042f3a0c8fb4de21bbf29: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 05:17:02.152463 30238 tablet_bootstrap.cc:492] T bf679b06ac034ca09579ad78fe5532dc P b405c1e0702e4a3786c9610471281200: No bootstrap required, opened a new log
I20260811 05:17:02.152534 30238 ts_tablet_manager.cc:1397] T bf679b06ac034ca09579ad78fe5532dc P b405c1e0702e4a3786c9610471281200: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 05:17:02.152766 30239 tablet_bootstrap.cc:654] T bf679b06ac034ca09579ad78fe5532dc P cf2fa555cbfa41e38d09bd1a962ced88: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:02.153302 30239 log.cc:824] T bf679b06ac034ca09579ad78fe5532dc P cf2fa555cbfa41e38d09bd1a962ced88: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:02.154296 30237 raft_consensus.cc:348] T bf679b06ac034ca09579ad78fe5532dc P 4f4fb338291042f3a0c8fb4de21bbf29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf2fa555cbfa41e38d09bd1a962ced88" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 36883 } } peers { permanent_uuid: "b405c1e0702e4a3786c9610471281200" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 36323 } } peers { permanent_uuid: "4f4fb338291042f3a0c8fb4de21bbf29" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36407 } }
I20260811 05:17:02.154359 30238 raft_consensus.cc:348] T bf679b06ac034ca09579ad78fe5532dc P b405c1e0702e4a3786c9610471281200 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf2fa555cbfa41e38d09bd1a962ced88" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 36883 } } peers { permanent_uuid: "b405c1e0702e4a3786c9610471281200" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 36323 } } peers { permanent_uuid: "4f4fb338291042f3a0c8fb4de21bbf29" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36407 } }
I20260811 05:17:02.154454 30237 raft_consensus.cc:374] T bf679b06ac034ca09579ad78fe5532dc P 4f4fb338291042f3a0c8fb4de21bbf29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:02.154491 30238 raft_consensus.cc:374] T bf679b06ac034ca09579ad78fe5532dc P b405c1e0702e4a3786c9610471281200 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:02.154493 30237 raft_consensus.cc:729] T bf679b06ac034ca09579ad78fe5532dc P 4f4fb338291042f3a0c8fb4de21bbf29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f4fb338291042f3a0c8fb4de21bbf29, State: Initialized, Role: FOLLOWER
I20260811 05:17:02.154533 30238 raft_consensus.cc:729] T bf679b06ac034ca09579ad78fe5532dc P b405c1e0702e4a3786c9610471281200 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b405c1e0702e4a3786c9610471281200, State: Initialized, Role: FOLLOWER
I20260811 05:17:02.154616 30237 consensus_queue.cc:260] T bf679b06ac034ca09579ad78fe5532dc P 4f4fb338291042f3a0c8fb4de21bbf29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf2fa555cbfa41e38d09bd1a962ced88" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 36883 } } peers { permanent_uuid: "b405c1e0702e4a3786c9610471281200" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 36323 } } peers { permanent_uuid: "4f4fb338291042f3a0c8fb4de21bbf29" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36407 } }
I20260811 05:17:02.154649 30238 consensus_queue.cc:260] T bf679b06ac034ca09579ad78fe5532dc P b405c1e0702e4a3786c9610471281200 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf2fa555cbfa41e38d09bd1a962ced88" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 36883 } } peers { permanent_uuid: "b405c1e0702e4a3786c9610471281200" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 36323 } } peers { permanent_uuid: "4f4fb338291042f3a0c8fb4de21bbf29" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36407 } }
I20260811 05:17:02.154891 30237 ts_tablet_manager.cc:1428] T bf679b06ac034ca09579ad78fe5532dc P 4f4fb338291042f3a0c8fb4de21bbf29: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 05:17:02.154937 30238 ts_tablet_manager.cc:1428] T bf679b06ac034ca09579ad78fe5532dc P b405c1e0702e4a3786c9610471281200: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260811 05:17:02.154978 29963 heartbeater.cc:503] Master 127.28.143.190:38797 was elected leader, sending a full tablet report...
I20260811 05:17:02.155051 29731 heartbeater.cc:503] Master 127.28.143.190:38797 was elected leader, sending a full tablet report...
I20260811 05:17:02.155673 30239 tablet_bootstrap.cc:492] T bf679b06ac034ca09579ad78fe5532dc P cf2fa555cbfa41e38d09bd1a962ced88: No bootstrap required, opened a new log
I20260811 05:17:02.155784 30239 ts_tablet_manager.cc:1397] T bf679b06ac034ca09579ad78fe5532dc P cf2fa555cbfa41e38d09bd1a962ced88: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260811 05:17:02.157636 30239 raft_consensus.cc:348] T bf679b06ac034ca09579ad78fe5532dc P cf2fa555cbfa41e38d09bd1a962ced88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf2fa555cbfa41e38d09bd1a962ced88" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 36883 } } peers { permanent_uuid: "b405c1e0702e4a3786c9610471281200" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 36323 } } peers { permanent_uuid: "4f4fb338291042f3a0c8fb4de21bbf29" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36407 } }
I20260811 05:17:02.157784 30239 raft_consensus.cc:374] T bf679b06ac034ca09579ad78fe5532dc P cf2fa555cbfa41e38d09bd1a962ced88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:02.157816 30239 raft_consensus.cc:729] T bf679b06ac034ca09579ad78fe5532dc P cf2fa555cbfa41e38d09bd1a962ced88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf2fa555cbfa41e38d09bd1a962ced88, State: Initialized, Role: FOLLOWER
I20260811 05:17:02.157934 30239 consensus_queue.cc:260] T bf679b06ac034ca09579ad78fe5532dc P cf2fa555cbfa41e38d09bd1a962ced88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf2fa555cbfa41e38d09bd1a962ced88" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 36883 } } peers { permanent_uuid: "b405c1e0702e4a3786c9610471281200" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 36323 } } peers { permanent_uuid: "4f4fb338291042f3a0c8fb4de21bbf29" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36407 } }
I20260811 05:17:02.158195 30239 ts_tablet_manager.cc:1428] T bf679b06ac034ca09579ad78fe5532dc P cf2fa555cbfa41e38d09bd1a962ced88: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 05:17:02.158267 30207 heartbeater.cc:503] Master 127.28.143.190:38797 was elected leader, sending a full tablet report...
W20260811 05:17:02.183305 29964 tablet.cc:2369] T bf679b06ac034ca09579ad78fe5532dc P b405c1e0702e4a3786c9610471281200: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 05:17:02.275885 29732 tablet.cc:2369] T bf679b06ac034ca09579ad78fe5532dc P 4f4fb338291042f3a0c8fb4de21bbf29: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 05:17:02.360112 30209 tablet.cc:2369] T bf679b06ac034ca09579ad78fe5532dc P cf2fa555cbfa41e38d09bd1a962ced88: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:17:02.382090 30249 raft_consensus.cc:482] T bf679b06ac034ca09579ad78fe5532dc P cf2fa555cbfa41e38d09bd1a962ced88 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:17:02.382239 30249 raft_consensus.cc:504] T bf679b06ac034ca09579ad78fe5532dc P cf2fa555cbfa41e38d09bd1a962ced88 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf2fa555cbfa41e38d09bd1a962ced88" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 36883 } } peers { permanent_uuid: "b405c1e0702e4a3786c9610471281200" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 36323 } } peers { permanent_uuid: "4f4fb338291042f3a0c8fb4de21bbf29" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36407 } }
I20260811 05:17:02.382673 30249 leader_election.cc:288] T bf679b06ac034ca09579ad78fe5532dc P cf2fa555cbfa41e38d09bd1a962ced88 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b405c1e0702e4a3786c9610471281200 (127.28.143.130:36323), 4f4fb338291042f3a0c8fb4de21bbf29 (127.28.143.129:36407)
I20260811 05:17:02.386929 30229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:02.382993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:55489 (local address 127.28.143.129:36407)
0811 05:17:02.383043 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:02.383045 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:02.383159 (+ 114us) server_negotiation.cc:407] Connection header received
0811 05:17:02.383207 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:02.383208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:02.383216 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:02.383261 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:02.383415 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:02.384146 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:02.385813 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:02.386391 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:02.386500 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:02.386511 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:02.386790 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 05:17:02.386850 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 05:17:02.386931 30232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:02.382999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:35267 (local address 127.28.143.130:36323)
0811 05:17:02.383043 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:02.383045 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:02.383056 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:17:02.383139 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:02.383140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:02.383149 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:02.383228 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:02.383390 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:02.384146 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:02.385813 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:02.386427 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:02.386500 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:02.386511 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:02.386789 (+ 278us) server_negotiation.cc:299] Negotiation successful
0811 05:17:02.386854 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 05:17:02.386916 30211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:02.382896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:36323 (local address 127.28.143.131:35267)
0811 05:17:02.383004 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:02.383018 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:02.383069 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:02.383240 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:02.383241 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:02.383242 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:02.383323 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:02.383325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:02.384309 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:02.384309 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:02.385668 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:02.385673 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:02.386541 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:02.386542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:02.386645 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:02.386648 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 05:17:02.386728 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 05:17:02.386784 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":36}
I20260811 05:17:02.386924 30251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:02.382907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:36407 (local address 127.28.143.131:55489)
0811 05:17:02.383092 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:02.383100 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:02.383152 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:02.383271 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:02.383272 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:02.383273 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:02.383324 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:02.383325 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:02.384292 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:02.384294 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:02.385668 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:02.385673 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:02.386541 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:02.386542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:02.386640 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:02.386646 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 05:17:02.386724 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 05:17:02.386784 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":69,"thread_start_us":101,"threads_started":1}
I20260811 05:17:02.387355 29881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf679b06ac034ca09579ad78fe5532dc" candidate_uuid: "cf2fa555cbfa41e38d09bd1a962ced88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b405c1e0702e4a3786c9610471281200" is_pre_election: true
I20260811 05:17:02.387359 29662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf679b06ac034ca09579ad78fe5532dc" candidate_uuid: "cf2fa555cbfa41e38d09bd1a962ced88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f4fb338291042f3a0c8fb4de21bbf29" is_pre_election: true
I20260811 05:17:02.387537 29881 raft_consensus.cc:2454] T bf679b06ac034ca09579ad78fe5532dc P b405c1e0702e4a3786c9610471281200 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2fa555cbfa41e38d09bd1a962ced88 in term 0.
I20260811 05:17:02.387656 29662 raft_consensus.cc:2454] T bf679b06ac034ca09579ad78fe5532dc P 4f4fb338291042f3a0c8fb4de21bbf29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf2fa555cbfa41e38d09bd1a962ced88 in term 0.
I20260811 05:17:02.387894 30000 leader_election.cc:302] T bf679b06ac034ca09579ad78fe5532dc P cf2fa555cbfa41e38d09bd1a962ced88 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b405c1e0702e4a3786c9610471281200, cf2fa555cbfa41e38d09bd1a962ced88; no voters:
I20260811 05:17:02.388055 30249 raft_consensus.cc:2781] T bf679b06ac034ca09579ad78fe5532dc P cf2fa555cbfa41e38d09bd1a962ced88 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:17:02.388119 30249 raft_consensus.cc:482] T bf679b06ac034ca09579ad78fe5532dc P cf2fa555cbfa41e38d09bd1a962ced88 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:17:02.388141 30249 raft_consensus.cc:3037] T bf679b06ac034ca09579ad78fe5532dc P cf2fa555cbfa41e38d09bd1a962ced88 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:02.388936 30249 raft_consensus.cc:504] T bf679b06ac034ca09579ad78fe5532dc P cf2fa555cbfa41e38d09bd1a962ced88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf2fa555cbfa41e38d09bd1a962ced88" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 36883 } } peers { permanent_uuid: "b405c1e0702e4a3786c9610471281200" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 36323 } } peers { permanent_uuid: "4f4fb338291042f3a0c8fb4de21bbf29" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36407 } }
I20260811 05:17:02.389181 30249 leader_election.cc:288] T bf679b06ac034ca09579ad78fe5532dc P cf2fa555cbfa41e38d09bd1a962ced88 [CANDIDATE]: Term 1 election: Requested vote from peers b405c1e0702e4a3786c9610471281200 (127.28.143.130:36323), 4f4fb338291042f3a0c8fb4de21bbf29 (127.28.143.129:36407)
I20260811 05:17:02.389416 29881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf679b06ac034ca09579ad78fe5532dc" candidate_uuid: "cf2fa555cbfa41e38d09bd1a962ced88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b405c1e0702e4a3786c9610471281200"
I20260811 05:17:02.389424 29662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf679b06ac034ca09579ad78fe5532dc" candidate_uuid: "cf2fa555cbfa41e38d09bd1a962ced88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f4fb338291042f3a0c8fb4de21bbf29"
I20260811 05:17:02.389516 29881 raft_consensus.cc:3037] T bf679b06ac034ca09579ad78fe5532dc P b405c1e0702e4a3786c9610471281200 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:02.389518 29662 raft_consensus.cc:3037] T bf679b06ac034ca09579ad78fe5532dc P 4f4fb338291042f3a0c8fb4de21bbf29 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:02.390539 29881 raft_consensus.cc:2454] T bf679b06ac034ca09579ad78fe5532dc P b405c1e0702e4a3786c9610471281200 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2fa555cbfa41e38d09bd1a962ced88 in term 1.
I20260811 05:17:02.390587 29662 raft_consensus.cc:2454] T bf679b06ac034ca09579ad78fe5532dc P 4f4fb338291042f3a0c8fb4de21bbf29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf2fa555cbfa41e38d09bd1a962ced88 in term 1.
I20260811 05:17:02.390769 30000 leader_election.cc:302] T bf679b06ac034ca09579ad78fe5532dc P cf2fa555cbfa41e38d09bd1a962ced88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b405c1e0702e4a3786c9610471281200, cf2fa555cbfa41e38d09bd1a962ced88; no voters:
I20260811 05:17:02.390882 30249 raft_consensus.cc:2781] T bf679b06ac034ca09579ad78fe5532dc P cf2fa555cbfa41e38d09bd1a962ced88 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:02.391177 30249 raft_consensus.cc:686] T bf679b06ac034ca09579ad78fe5532dc P cf2fa555cbfa41e38d09bd1a962ced88 [term 1 LEADER]: Becoming Leader. State: Replica: cf2fa555cbfa41e38d09bd1a962ced88, State: Running, Role: LEADER
I20260811 05:17:02.391322 30249 consensus_queue.cc:237] T bf679b06ac034ca09579ad78fe5532dc P cf2fa555cbfa41e38d09bd1a962ced88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf2fa555cbfa41e38d09bd1a962ced88" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 36883 } } peers { permanent_uuid: "b405c1e0702e4a3786c9610471281200" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 36323 } } peers { permanent_uuid: "4f4fb338291042f3a0c8fb4de21bbf29" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36407 } }
I20260811 05:17:02.392750 29419 catalog_manager.cc:5495] T bf679b06ac034ca09579ad78fe5532dc P cf2fa555cbfa41e38d09bd1a962ced88 reported cstate change: term changed from 0 to 1, leader changed from <none> to cf2fa555cbfa41e38d09bd1a962ced88 (127.28.143.131). New cstate: current_term: 1 leader_uuid: "cf2fa555cbfa41e38d09bd1a962ced88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf2fa555cbfa41e38d09bd1a962ced88" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 36883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b405c1e0702e4a3786c9610471281200" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 36323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f4fb338291042f3a0c8fb4de21bbf29" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36407 } health_report { overall_health: UNKNOWN } } }
I20260811 05:17:02.434847 30233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:02.432079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56644 (local address 127.28.143.131:36883)
0811 05:17:02.432136 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:02.432138 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:02.432147 (+ 9us) server_negotiation.cc:407] Connection header received
0811 05:17:02.432186 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:02.432187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:02.432206 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:02.432289 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:17:02.432516 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:02.433070 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:02.433952 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:02.434166 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:02.434221 (+ 55us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:02.434455 (+ 234us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:17:02.434560 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:17:02.434634 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:02.434744 (+ 110us) server_negotiation.cc:299] Negotiation successful
0811 05:17:02.434757 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 05:17:02.436892 29662 raft_consensus.cc:1261] T bf679b06ac034ca09579ad78fe5532dc P 4f4fb338291042f3a0c8fb4de21bbf29 [term 1 FOLLOWER]: Refusing update from remote peer cf2fa555cbfa41e38d09bd1a962ced88: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:17:02.436892 29881 raft_consensus.cc:1261] T bf679b06ac034ca09579ad78fe5532dc P b405c1e0702e4a3786c9610471281200 [term 1 FOLLOWER]: Refusing update from remote peer cf2fa555cbfa41e38d09bd1a962ced88: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:17:02.437283 30252 consensus_queue.cc:1035] T bf679b06ac034ca09579ad78fe5532dc P cf2fa555cbfa41e38d09bd1a962ced88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f4fb338291042f3a0c8fb4de21bbf29" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 36407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:17:02.437434 30249 consensus_queue.cc:1035] T bf679b06ac034ca09579ad78fe5532dc P cf2fa555cbfa41e38d09bd1a962ced88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b405c1e0702e4a3786c9610471281200" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 36323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:17:02.441010 30261 mvcc.cc:205] Tried to move back new op lower bound from 7317198530298466304 to 7317198530116157440. Current Snapshot: MvccSnapshot[applied={T|T < 7317198530298466304 or (T in {7317198530298466304})}]
I20260811 05:17:02.441160 30258 mvcc.cc:205] Tried to move back new op lower bound from 7317198530298466304 to 7317198530116157440. Current Snapshot: MvccSnapshot[applied={T|T < 7317198530298466304 or (T in {7317198530298466304})}]
I20260811 05:17:02.443212 29419 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:17:02.443322 29419 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:17:02.444231 29419 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1c8fb3a494ff49b294e14435bc7f970e: Sending DeleteTablet for 3 replicas of tablet bf679b06ac034ca09579ad78fe5532dc
I20260811 05:17:02.444643 30085 tablet_service.cc:1514] Processing DeleteTablet for tablet bf679b06ac034ca09579ad78fe5532dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:17:02 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:56634
I20260811 05:17:02.444701 29849 tablet_service.cc:1514] Processing DeleteTablet for tablet bf679b06ac034ca09579ad78fe5532dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:17:02 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:35522
I20260811 05:17:02.444799 29626 tablet_service.cc:1514] Processing DeleteTablet for tablet bf679b06ac034ca09579ad78fe5532dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:17:02 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:51094
I20260811 05:17:02.444890 30286 tablet_replica.cc:330] T bf679b06ac034ca09579ad78fe5532dc P b405c1e0702e4a3786c9610471281200: stopping tablet replica
I20260811 05:17:02.444983 30286 raft_consensus.cc:2229] T bf679b06ac034ca09579ad78fe5532dc P b405c1e0702e4a3786c9610471281200 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:17:02.445098 30286 raft_consensus.cc:2258] T bf679b06ac034ca09579ad78fe5532dc P b405c1e0702e4a3786c9610471281200 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:17:02.445106 30288 tablet_replica.cc:330] T bf679b06ac034ca09579ad78fe5532dc P 4f4fb338291042f3a0c8fb4de21bbf29: stopping tablet replica
I20260811 05:17:02.445210 30288 raft_consensus.cc:2229] T bf679b06ac034ca09579ad78fe5532dc P 4f4fb338291042f3a0c8fb4de21bbf29 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:17:02.445477 30288 raft_consensus.cc:2258] T bf679b06ac034ca09579ad78fe5532dc P 4f4fb338291042f3a0c8fb4de21bbf29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:17:02.445708 30286 ts_tablet_manager.cc:1905] T bf679b06ac034ca09579ad78fe5532dc P b405c1e0702e4a3786c9610471281200: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:17:02.445924 30287 tablet_replica.cc:330] T bf679b06ac034ca09579ad78fe5532dc P cf2fa555cbfa41e38d09bd1a962ced88: stopping tablet replica
I20260811 05:17:02.445936 30288 ts_tablet_manager.cc:1905] T bf679b06ac034ca09579ad78fe5532dc P 4f4fb338291042f3a0c8fb4de21bbf29: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:17:02.446099 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 29524
I20260811 05:17:02.446084 30287 raft_consensus.cc:2229] T bf679b06ac034ca09579ad78fe5532dc P cf2fa555cbfa41e38d09bd1a962ced88 [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:17:02.446364 30287 raft_consensus.cc:2258] T bf679b06ac034ca09579ad78fe5532dc P cf2fa555cbfa41e38d09bd1a962ced88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:17:02.451180 30286 ts_tablet_manager.cc:1918] T bf679b06ac034ca09579ad78fe5532dc P b405c1e0702e4a3786c9610471281200: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:17:02.451408 30286 log.cc:1196] T bf679b06ac034ca09579ad78fe5532dc P b405c1e0702e4a3786c9610471281200: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-1/wal/wals/bf679b06ac034ca09579ad78fe5532dc
I20260811 05:17:02.452355 30286 ts_tablet_manager.cc:1939] T bf679b06ac034ca09579ad78fe5532dc P b405c1e0702e4a3786c9610471281200: Deleting consensus metadata
I20260811 05:17:02.453192 29390 catalog_manager.cc:4841] TS b405c1e0702e4a3786c9610471281200 (127.28.143.130:36323): tablet bf679b06ac034ca09579ad78fe5532dc (table test-table [id=903db0e422be487ba87b2be4bef67e66]) successfully deleted
I20260811 05:17:02.454111 30287 ts_tablet_manager.cc:1905] T bf679b06ac034ca09579ad78fe5532dc P cf2fa555cbfa41e38d09bd1a962ced88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:17:02.456310 30287 ts_tablet_manager.cc:1918] T bf679b06ac034ca09579ad78fe5532dc P cf2fa555cbfa41e38d09bd1a962ced88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:17:02.456434 30287 log.cc:1196] T bf679b06ac034ca09579ad78fe5532dc P cf2fa555cbfa41e38d09bd1a962ced88: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425159060450-29246-0/minicluster-data/ts-2/wal/wals/bf679b06ac034ca09579ad78fe5532dc
I20260811 05:17:02.456806 30287 ts_tablet_manager.cc:1939] T bf679b06ac034ca09579ad78fe5532dc P cf2fa555cbfa41e38d09bd1a962ced88: Deleting consensus metadata
I20260811 05:17:02.457427 29392 catalog_manager.cc:4841] TS cf2fa555cbfa41e38d09bd1a962ced88 (127.28.143.131:36883): tablet bf679b06ac034ca09579ad78fe5532dc (table test-table [id=903db0e422be487ba87b2be4bef67e66]) successfully deleted
W20260811 05:17:02.459080 29394 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.143.129:36407 (error 108)
I20260811 05:17:02.459718 30228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:02.459567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:36407 (local address 127.0.0.1:51100)
0811 05:17:02.459631 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:02.459678 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.129:36407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 05:17:02.459748 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 29738
W20260811 05:17:02.459873 29394 catalog_manager.cc:4584] TS 4f4fb338291042f3a0c8fb4de21bbf29 (127.28.143.129:36407): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bf679b06ac034ca09579ad78fe5532dc: Network error: Client connection negotiation failed: client connection to 127.28.143.129:36407: connect: Connection refused (error 111)
I20260811 05:17:02.472980 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 29972
I20260811 05:17:02.493773 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 29360
2026-08-11T05:17:02Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2426 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T05:17:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:17:02Z Disabled control of system clock
I20260811 05:17:02.535017 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:46411
--webserver_interface=127.28.143.190
--webserver_port=0
--builtin_ntp_servers=127.28.143.148:39639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:46411
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 05:17:02.620306 30296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:02.620544 30296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:02.620564 30296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:02.621766 30296 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:02.621819 30296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:02.621830 30296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:02.621842 30296 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:02.621853 30296 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:02.623263 30296 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:39639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:46411
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:46411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30296
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:17:02.623498 30296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:02.623785 30296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:02.629369 30309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:02.629352 30311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:02.629400 30296 server_base.cc:1034] running on GCE node
W20260811 05:17:02.629352 30302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:02.629870 30296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:02.630220 30296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:02.631402 30296 hybrid_clock.cc:648] HybridClock initialized: now 1786425422631380 us; error 38 us; skew 500 ppm
I20260811 05:17:02.632855 30296 webserver.cc:460] Webserver started at http://127.28.143.190:45225/ using document root <none> and password file <none>
I20260811 05:17:02.633159 30296 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:02.633219 30296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:02.633354 30296 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:02.634465 30296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "07ab75168df3490ea87c7bd15e14c8bd"
format_stamp: "Formatted at 2026-08-11 05:17:02 on dist-test-slave-f7th"
server_key: "84ad6826f0822be9c881fc99a30b6cb5"
server_key_iv: "35aba7ae85ed8653a6f365081a802f8d"
server_key_version: "encryptionkey@0"
I20260811 05:17:02.634804 30296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "07ab75168df3490ea87c7bd15e14c8bd"
format_stamp: "Formatted at 2026-08-11 05:17:02 on dist-test-slave-f7th"
server_key: "84ad6826f0822be9c881fc99a30b6cb5"
server_key_iv: "35aba7ae85ed8653a6f365081a802f8d"
server_key_version: "encryptionkey@0"
I20260811 05:17:02.636240 30296 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 05:17:02.637128 30322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:02.637290 30296 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 05:17:02.637384 30296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "07ab75168df3490ea87c7bd15e14c8bd"
format_stamp: "Formatted at 2026-08-11 05:17:02 on dist-test-slave-f7th"
server_key: "84ad6826f0822be9c881fc99a30b6cb5"
server_key_iv: "35aba7ae85ed8653a6f365081a802f8d"
server_key_version: "encryptionkey@0"
I20260811 05:17:02.637451 30296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:02.651705 30296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:02.652078 30296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:02.652221 30296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:02.656792 30296 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:46411
I20260811 05:17:02.656909 30426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:46411 every 8 connection(s)
I20260811 05:17:02.657217 30296 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:17:02.658305 30427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:02.660290 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 30296
I20260811 05:17:02.660385 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:17:02.660581 29246 external_mini_cluster.cc:1428] Setting key ae87420cdaa801c3e2abd6b38921469f
I20260811 05:17:02.660789 30427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07ab75168df3490ea87c7bd15e14c8bd: Bootstrap starting.
I20260811 05:17:02.661530 30427 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 07ab75168df3490ea87c7bd15e14c8bd: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:02.661820 30427 log.cc:824] T 00000000000000000000000000000000 P 07ab75168df3490ea87c7bd15e14c8bd: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:02.662528 30427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07ab75168df3490ea87c7bd15e14c8bd: No bootstrap required, opened a new log
I20260811 05:17:02.664283 30427 raft_consensus.cc:348] T 00000000000000000000000000000000 P 07ab75168df3490ea87c7bd15e14c8bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07ab75168df3490ea87c7bd15e14c8bd" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 46411 } }
I20260811 05:17:02.664424 30427 raft_consensus.cc:374] T 00000000000000000000000000000000 P 07ab75168df3490ea87c7bd15e14c8bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:02.664456 30427 raft_consensus.cc:729] T 00000000000000000000000000000000 P 07ab75168df3490ea87c7bd15e14c8bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07ab75168df3490ea87c7bd15e14c8bd, State: Initialized, Role: FOLLOWER
I20260811 05:17:02.664669 30427 consensus_queue.cc:260] T 00000000000000000000000000000000 P 07ab75168df3490ea87c7bd15e14c8bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07ab75168df3490ea87c7bd15e14c8bd" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 46411 } }
I20260811 05:17:02.664772 30427 raft_consensus.cc:388] T 00000000000000000000000000000000 P 07ab75168df3490ea87c7bd15e14c8bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:17:02.664808 30427 raft_consensus.cc:482] T 00000000000000000000000000000000 P 07ab75168df3490ea87c7bd15e14c8bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:17:02.664857 30427 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 07ab75168df3490ea87c7bd15e14c8bd [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:02.665540 30427 raft_consensus.cc:504] T 00000000000000000000000000000000 P 07ab75168df3490ea87c7bd15e14c8bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07ab75168df3490ea87c7bd15e14c8bd" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 46411 } }
I20260811 05:17:02.665654 30427 leader_election.cc:302] T 00000000000000000000000000000000 P 07ab75168df3490ea87c7bd15e14c8bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 07ab75168df3490ea87c7bd15e14c8bd; no voters:
I20260811 05:17:02.665856 30427 leader_election.cc:288] T 00000000000000000000000000000000 P 07ab75168df3490ea87c7bd15e14c8bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:17:02.665935 30435 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 07ab75168df3490ea87c7bd15e14c8bd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:02.666049 30432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:02.661161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56994 (local address 127.28.143.190:46411)
0811 05:17:02.661371 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:02.661378 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:02.661404 (+ 26us) server_negotiation.cc:407] Connection header received
0811 05:17:02.661883 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:02.661890 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:02.661972 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:02.662108 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:02.662341 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:02.663281 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:02.664059 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:02.664479 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:02.664598 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:02.665292 (+ 694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:02.665297 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:02.665297 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:02.665478 (+ 181us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:02.665513 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:02.665525 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:02.665574 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:02.665685 (+ 111us) server_negotiation.cc:299] Negotiation successful
0811 05:17:02.665782 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":74,"threads_started":1}
I20260811 05:17:02.666085 30435 raft_consensus.cc:686] T 00000000000000000000000000000000 P 07ab75168df3490ea87c7bd15e14c8bd [term 1 LEADER]: Becoming Leader. State: Replica: 07ab75168df3490ea87c7bd15e14c8bd, State: Running, Role: LEADER
I20260811 05:17:02.666185 30427 sys_catalog.cc:564] T 00000000000000000000000000000000 P 07ab75168df3490ea87c7bd15e14c8bd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:02.666419 30435 consensus_queue.cc:237] T 00000000000000000000000000000000 P 07ab75168df3490ea87c7bd15e14c8bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07ab75168df3490ea87c7bd15e14c8bd" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 46411 } }
I20260811 05:17:02.667050 30438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07ab75168df3490ea87c7bd15e14c8bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 07ab75168df3490ea87c7bd15e14c8bd. Latest consensus state: current_term: 1 leader_uuid: "07ab75168df3490ea87c7bd15e14c8bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07ab75168df3490ea87c7bd15e14c8bd" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 46411 } } }
I20260811 05:17:02.667059 30437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07ab75168df3490ea87c7bd15e14c8bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "07ab75168df3490ea87c7bd15e14c8bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07ab75168df3490ea87c7bd15e14c8bd" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 46411 } } }
I20260811 05:17:02.667155 30438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07ab75168df3490ea87c7bd15e14c8bd [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:02.667158 30437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07ab75168df3490ea87c7bd15e14c8bd [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:02.667367 30451 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:17:02.667975 30451 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:17:02.669427 30451 catalog_manager.cc:1347] Generated new cluster ID: 11621ab751e94aa789cf81c85ff0e222
I20260811 05:17:02.669482 30451 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:17:02.682621 30451 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:17:02.683230 30451 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:17:02.687861 30451 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 07ab75168df3490ea87c7bd15e14c8bd: Generated new TSK 0
I20260811 05:17:02.688119 30451 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:17:02.691612 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.129:0
--local_ip_for_outbound_sockets=127.28.143.129
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:46411
--builtin_ntp_servers=127.28.143.148:39639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 05:17:02.782987 30462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:02.783231 30462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:02.783255 30462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:02.784569 30462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:02.784663 30462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:02.784730 30462 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.129
I20260811 05:17:02.786355 30462 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:39639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:46411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.129
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:17:02.786648 30462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:02.787025 30462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:02.792202 30473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:02.792217 30482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:02.792232 30474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:02.792573 30462 server_base.cc:1034] running on GCE node
I20260811 05:17:02.792810 30462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:02.793056 30462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:02.794225 30462 hybrid_clock.cc:648] HybridClock initialized: now 1786425422794210 us; error 30 us; skew 500 ppm
I20260811 05:17:02.795627 30462 webserver.cc:460] Webserver started at http://127.28.143.129:43617/ using document root <none> and password file <none>
I20260811 05:17:02.795892 30462 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:02.795959 30462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:02.796099 30462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:02.797250 30462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-0/data/instance:
uuid: "5361647f33364ecaa92a8acff75e9d1f"
format_stamp: "Formatted at 2026-08-11 05:17:02 on dist-test-slave-f7th"
server_key: "06a7cf1a65178a642fc04e7017139064"
server_key_iv: "c3d1eef2b5c666ff8099006899384f75"
server_key_version: "encryptionkey@0"
I20260811 05:17:02.797605 30462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance:
uuid: "5361647f33364ecaa92a8acff75e9d1f"
format_stamp: "Formatted at 2026-08-11 05:17:02 on dist-test-slave-f7th"
server_key: "06a7cf1a65178a642fc04e7017139064"
server_key_iv: "c3d1eef2b5c666ff8099006899384f75"
server_key_version: "encryptionkey@0"
I20260811 05:17:02.799036 30462 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 05:17:02.799834 30496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:02.800015 30462 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:17:02.800118 30462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-0/wal
uuid: "5361647f33364ecaa92a8acff75e9d1f"
format_stamp: "Formatted at 2026-08-11 05:17:02 on dist-test-slave-f7th"
server_key: "06a7cf1a65178a642fc04e7017139064"
server_key_iv: "c3d1eef2b5c666ff8099006899384f75"
server_key_version: "encryptionkey@0"
I20260811 05:17:02.800187 30462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:02.814668 30462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:02.815003 30462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:02.815140 30462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:02.815371 30462 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:17:02.815723 30462 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:17:02.815758 30462 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:02.815788 30462 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:17:02.815812 30462 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:02.822170 30462 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.129:37261
I20260811 05:17:02.822216 30692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.129:37261 every 8 connection(s)
I20260811 05:17:02.822571 30462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
I20260811 05:17:02.828158 30432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:02.823334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:58153 (local address 127.28.143.190:46411)
0811 05:17:02.823417 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:02.823419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:02.823663 (+ 244us) server_negotiation.cc:407] Connection header received
0811 05:17:02.824131 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:02.824133 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:02.824144 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:02.824201 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:02.824994 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:02.825522 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:02.826561 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:02.826757 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:02.826825 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:02.827698 (+ 873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:02.827700 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:02.827700 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:02.827705 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:02.827720 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:02.827724 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:02.827772 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:02.828034 (+ 262us) server_negotiation.cc:299] Negotiation successful
0811 05:17:02.828068 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 05:17:02.828382 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 30462
I20260811 05:17:02.828482 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance
I20260811 05:17:02.828532 30697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:02.823299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:46411 (local address 127.28.143.129:58153)
0811 05:17:02.823512 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:02.823550 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:02.824039 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:02.824269 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:02.824272 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:02.824281 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:02.824892 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:02.824898 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:02.825607 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:02.825608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:02.826464 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:02.826467 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:02.826856 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:02.826857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:02.826985 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:02.827546 (+ 561us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:17:02.827547 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:02.827577 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:17:02.827601 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:02.827851 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:02.827863 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:02.827867 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:17:02.827996 (+ 129us) client_negotiation.cc:240] Negotiation successful
0811 05:17:02.828154 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":88,"threads_started":1}
I20260811 05:17:02.828730 29246 external_mini_cluster.cc:1428] Setting key 2c8de5304f3da04e05ea645a3d39ba4e
I20260811 05:17:02.829303 30694 heartbeater.cc:347] Connected to a master server at 127.28.143.190:46411
I20260811 05:17:02.829416 30694 heartbeater.cc:464] Registering TS with master...
I20260811 05:17:02.829630 30694 heartbeater.cc:511] Master 127.28.143.190:46411 requested a full tablet report, sending...
I20260811 05:17:02.829744 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.130:0
--local_ip_for_outbound_sockets=127.28.143.130
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:46411
--builtin_ntp_servers=127.28.143.148:39639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 05:17:02.831817 30356 ts_manager.cc:194] Registered new tserver with Master: 5361647f33364ecaa92a8acff75e9d1f (127.28.143.129:37261)
I20260811 05:17:02.832778 30356 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.143.129:58153
W20260811 05:17:02.916806 30698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:02.917035 30698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:02.917068 30698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:02.918252 30698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:02.918295 30698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:02.918329 30698 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.130
I20260811 05:17:02.919720 30698 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:39639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:46411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30698
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.130
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:17:02.919979 30698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:02.920297 30698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:02.925992 30705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:02.925998 30710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:02.926110 30698 server_base.cc:1034] running on GCE node
W20260811 05:17:02.926004 30715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:02.926455 30698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:02.926698 30698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:02.927939 30698 hybrid_clock.cc:648] HybridClock initialized: now 1786425422927924 us; error 30 us; skew 500 ppm
I20260811 05:17:02.929380 30698 webserver.cc:460] Webserver started at http://127.28.143.130:46013/ using document root <none> and password file <none>
I20260811 05:17:02.929658 30698 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:02.929708 30698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:02.929834 30698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:02.930941 30698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-1/data/instance:
uuid: "ee7547adbaa443409bcd9482a2891d55"
format_stamp: "Formatted at 2026-08-11 05:17:02 on dist-test-slave-f7th"
server_key: "2898d7aa5bdd2205118a8a22a83084b8"
server_key_iv: "68d628a75e1f849f08e2725ac96ded83"
server_key_version: "encryptionkey@0"
I20260811 05:17:02.931293 30698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance:
uuid: "ee7547adbaa443409bcd9482a2891d55"
format_stamp: "Formatted at 2026-08-11 05:17:02 on dist-test-slave-f7th"
server_key: "2898d7aa5bdd2205118a8a22a83084b8"
server_key_iv: "68d628a75e1f849f08e2725ac96ded83"
server_key_version: "encryptionkey@0"
I20260811 05:17:02.932560 30698 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 05:17:02.933350 30725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:02.933506 30698 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 05:17:02.933604 30698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-1/wal
uuid: "ee7547adbaa443409bcd9482a2891d55"
format_stamp: "Formatted at 2026-08-11 05:17:02 on dist-test-slave-f7th"
server_key: "2898d7aa5bdd2205118a8a22a83084b8"
server_key_iv: "68d628a75e1f849f08e2725ac96ded83"
server_key_version: "encryptionkey@0"
I20260811 05:17:02.933671 30698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:02.965946 30698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:02.966300 30698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:02.966454 30698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:02.966741 30698 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:17:02.967149 30698 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:17:02.967185 30698 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:02.967207 30698 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:17:02.967231 30698 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:02.972960 30698 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.130:33487
I20260811 05:17:02.973024 30933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.130:33487 every 8 connection(s)
I20260811 05:17:02.973364 30698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
I20260811 05:17:02.977018 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 30698
I20260811 05:17:02.977109 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance
I20260811 05:17:02.977300 29246 external_mini_cluster.cc:1428] Setting key 02b2fd8071f7082f3ba0a008821aae92
I20260811 05:17:02.978289 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.131:0
--local_ip_for_outbound_sockets=127.28.143.131
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:46411
--builtin_ntp_servers=127.28.143.148:39639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 05:17:02.978966 30432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:02.974176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:51217 (local address 127.28.143.190:46411)
0811 05:17:02.974251 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:02.974253 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:02.974486 (+ 233us) server_negotiation.cc:407] Connection header received
0811 05:17:02.975007 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:02.975008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:02.975019 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:02.975068 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:02.975791 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:02.976313 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:02.977519 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:02.977712 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:02.977765 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:02.978719 (+ 954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:02.978720 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:02.978721 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:02.978725 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:02.978742 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:02.978747 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:02.978791 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:02.978880 (+ 89us) server_negotiation.cc:299] Negotiation successful
0811 05:17:02.978904 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 05:17:02.979236 30938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:02.974143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:46411 (local address 127.28.143.130:51217)
0811 05:17:02.974360 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:02.974398 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:02.974921 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:02.975140 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:02.975143 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:02.975152 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:02.975679 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:02.975683 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:02.976432 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:02.976433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:02.977402 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:02.977405 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:02.977785 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:02.977786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:02.977942 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:02.978547 (+ 605us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:17:02.978548 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:02.978589 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:17:02.978624 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:02.978805 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:02.978811 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:02.978814 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 05:17:02.978882 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 05:17:02.979004 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":88,"threads_started":1}
I20260811 05:17:02.979702 30934 heartbeater.cc:347] Connected to a master server at 127.28.143.190:46411
I20260811 05:17:02.979799 30934 heartbeater.cc:464] Registering TS with master...
I20260811 05:17:02.980006 30934 heartbeater.cc:511] Master 127.28.143.190:46411 requested a full tablet report, sending...
I20260811 05:17:02.980472 30356 ts_manager.cc:194] Registered new tserver with Master: ee7547adbaa443409bcd9482a2891d55 (127.28.143.130:33487)
I20260811 05:17:02.980994 30356 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.143.130:51217
W20260811 05:17:03.064976 30940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:03.065204 30940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:03.065229 30940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:03.066624 30940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:03.066694 30940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:03.066730 30940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.131
I20260811 05:17:03.068348 30940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:39639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:46411
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.131
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:17:03.068609 30940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:03.068943 30940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:03.074491 30948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:03.074596 30940 server_base.cc:1034] running on GCE node
W20260811 05:17:03.074525 30947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:03.074491 30951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:03.075086 30940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:03.075361 30940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:03.076535 30940 hybrid_clock.cc:648] HybridClock initialized: now 1786425423076512 us; error 33 us; skew 500 ppm
I20260811 05:17:03.078038 30940 webserver.cc:460] Webserver started at http://127.28.143.131:32793/ using document root <none> and password file <none>
I20260811 05:17:03.078320 30940 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:03.078372 30940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:03.078521 30940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:03.079655 30940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-2/data/instance:
uuid: "ac12075ba1b44514881e1db1e42c4594"
format_stamp: "Formatted at 2026-08-11 05:17:03 on dist-test-slave-f7th"
server_key: "5f34868960d5f094430dc6f81eccf85c"
server_key_iv: "7eb10763e259b5cd692e64105821e450"
server_key_version: "encryptionkey@0"
I20260811 05:17:03.079998 30940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance:
uuid: "ac12075ba1b44514881e1db1e42c4594"
format_stamp: "Formatted at 2026-08-11 05:17:03 on dist-test-slave-f7th"
server_key: "5f34868960d5f094430dc6f81eccf85c"
server_key_iv: "7eb10763e259b5cd692e64105821e450"
server_key_version: "encryptionkey@0"
I20260811 05:17:03.081382 30940 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 05:17:03.082281 30963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:03.082458 30940 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:17:03.082563 30940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-2/wal
uuid: "ac12075ba1b44514881e1db1e42c4594"
format_stamp: "Formatted at 2026-08-11 05:17:03 on dist-test-slave-f7th"
server_key: "5f34868960d5f094430dc6f81eccf85c"
server_key_iv: "7eb10763e259b5cd692e64105821e450"
server_key_version: "encryptionkey@0"
I20260811 05:17:03.082641 30940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:03.102504 30940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:03.102838 30940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:03.102973 30940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:03.103221 30940 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:17:03.103609 30940 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:17:03.103649 30940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:03.103679 30940 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:17:03.103696 30940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:03.109272 30940 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.131:32801
I20260811 05:17:03.109323 31173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.131:32801 every 8 connection(s)
I20260811 05:17:03.109665 30940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
I20260811 05:17:03.115118 30432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:03.110469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:59783 (local address 127.28.143.190:46411)
0811 05:17:03.110532 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:03.110533 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:03.110713 (+ 180us) server_negotiation.cc:407] Connection header received
0811 05:17:03.111217 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:03.111218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:03.111230 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:03.111296 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:03.111983 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:03.112506 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:03.113587 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:03.113783 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:03.113857 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:03.114792 (+ 935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:03.114794 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:03.114794 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:03.114799 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:03.114817 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:03.114821 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:03.114868 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:03.115023 (+ 155us) server_negotiation.cc:299] Negotiation successful
0811 05:17:03.115051 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 05:17:03.115267 31180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:03.110383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:46411 (local address 127.28.143.131:59783)
0811 05:17:03.110585 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:03.110629 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:03.111109 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:03.111361 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:03.111365 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:03.111373 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:03.111886 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:03.111891 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:03.112595 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:03.112596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:03.113475 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:03.113478 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:03.113929 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:03.113930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:03.114062 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:03.114636 (+ 574us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:17:03.114637 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:03.114670 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:17:03.114695 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:03.114916 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:03.114922 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:03.114925 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 05:17:03.114981 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 05:17:03.115071 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":81,"threads_started":1}
I20260811 05:17:03.115634 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 30940
I20260811 05:17:03.115736 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance
I20260811 05:17:03.115938 29246 external_mini_cluster.cc:1428] Setting key 751eaca34affdabe6927ecd234e6d276
I20260811 05:17:03.115932 31176 heartbeater.cc:347] Connected to a master server at 127.28.143.190:46411
I20260811 05:17:03.116043 31176 heartbeater.cc:464] Registering TS with master...
I20260811 05:17:03.116278 31176 heartbeater.cc:511] Master 127.28.143.190:46411 requested a full tablet report, sending...
I20260811 05:17:03.116734 30355 ts_manager.cc:194] Registered new tserver with Master: ac12075ba1b44514881e1db1e42c4594 (127.28.143.131:32801)
I20260811 05:17:03.117174 30355 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.143.131:59783
I20260811 05:17:03.128281 29246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:17:03.132876 30432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:03.129772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56998 (local address 127.28.143.190:46411)
0811 05:17:03.129841 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:03.129843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:03.129920 (+ 77us) server_negotiation.cc:407] Connection header received
0811 05:17:03.129980 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:03.129981 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:03.129992 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:03.130042 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:03.130209 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:03.130740 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:03.131545 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:03.131732 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:03.131789 (+ 57us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:03.132543 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:03.132544 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:03.132545 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:03.132550 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:03.132564 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:03.132568 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:03.132697 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:03.132769 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 05:17:03.132782 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 05:17:03.134796 30355 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56998:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:17:03.136343 30355 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:17:03.143018 31193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:03.138619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.131:32801 (local address 127.0.0.1:37360)
0811 05:17:03.138794 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:03.138833 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:03.138898 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:03.139573 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:03.139574 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:03.139575 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:03.139620 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:03.139622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:03.140542 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:03.140543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:03.141827 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:03.141831 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:03.142692 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:03.142693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:03.142772 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:03.142786 (+ 14us) client_negotiation.cc:769] Sending connection context
0811 05:17:03.142858 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 05:17:03.142916 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":79,"thread_start_us":75,"threads_started":1}
I20260811 05:17:03.143011 31202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:03.138863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37360 (local address 127.28.143.131:32801)
0811 05:17:03.139363 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:03.139367 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:03.139376 (+ 9us) server_negotiation.cc:407] Connection header received
0811 05:17:03.139411 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:03.139413 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:03.139427 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:03.139507 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:03.139695 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:03.140471 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:03.141910 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:03.142596 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:03.142676 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:03.142769 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:03.142871 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 05:17:03.142911 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":99,"threads_started":1}
I20260811 05:17:03.143124 31200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:03.138869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40138 (local address 127.28.143.130:33487)
0811 05:17:03.139082 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:03.139086 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:03.139095 (+ 9us) server_negotiation.cc:407] Connection header received
0811 05:17:03.139130 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:03.139132 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:03.139147 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:03.139239 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:03.139430 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:03.140577 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:03.142169 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:03.142678 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:03.142747 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:03.142833 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:03.143011 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 05:17:03.143051 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":73,"threads_started":1}
I20260811 05:17:03.143224 31198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:03.138619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:37261 (local address 127.0.0.1:42702)
0811 05:17:03.139097 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:03.139104 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:03.139140 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:03.139307 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:03.139308 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:03.139309 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:03.139343 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:03.139344 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:03.140687 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:03.140688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:03.142026 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:03.142029 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:03.143001 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:03.143003 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:03.143059 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:03.143064 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 05:17:03.143113 (+ 49us) client_negotiation.cc:240] Negotiation successful
0811 05:17:03.143156 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"mutex_wait_us":23,"thread_start_us":90,"threads_started":1}
I20260811 05:17:03.143496 31199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:03.138797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42702 (local address 127.28.143.129:37261)
0811 05:17:03.139010 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:03.139016 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:03.139159 (+ 143us) server_negotiation.cc:407] Connection header received
0811 05:17:03.139204 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:03.139206 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:03.139220 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:03.139293 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:03.139429 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:03.140577 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:03.142118 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:03.142816 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:03.142879 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:03.142963 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:03.143364 (+ 401us) server_negotiation.cc:299] Negotiation successful
0811 05:17:03.143409 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":102,"threads_started":1}
I20260811 05:17:03.143824 31196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:03.138620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:33487 (local address 127.0.0.1:40138)
0811 05:17:03.138991 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:03.138999 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:03.139033 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:03.139241 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:03.139243 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:03.139245 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:03.139316 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:03.139318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:03.140709 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:03.140710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:03.142105 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:03.142107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:03.142868 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:03.142869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:03.142921 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:03.142923 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 05:17:03.143706 (+ 783us) client_negotiation.cc:240] Negotiation successful
0811 05:17:03.143756 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"mutex_wait_us":83,"thread_start_us":66,"threads_started":1}
I20260811 05:17:03.143899 31069 tablet_service.cc:1467] Processing CreateTablet for tablet 6725de54cf924f959a51c59d3cd06af8 (DEFAULT_TABLE table=test-table [id=7bd6ac0d14c0496b86982caf90749a5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:17:03.144114 30579 tablet_service.cc:1467] Processing CreateTablet for tablet 6725de54cf924f959a51c59d3cd06af8 (DEFAULT_TABLE table=test-table [id=7bd6ac0d14c0496b86982caf90749a5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:17:03.144325 31069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6725de54cf924f959a51c59d3cd06af8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:03.144482 30579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6725de54cf924f959a51c59d3cd06af8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:03.145205 30804 tablet_service.cc:1467] Processing CreateTablet for tablet 6725de54cf924f959a51c59d3cd06af8 (DEFAULT_TABLE table=test-table [id=7bd6ac0d14c0496b86982caf90749a5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:17:03.145578 30804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6725de54cf924f959a51c59d3cd06af8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:03.147188 31204 tablet_bootstrap.cc:492] T 6725de54cf924f959a51c59d3cd06af8 P 5361647f33364ecaa92a8acff75e9d1f: Bootstrap starting.
I20260811 05:17:03.147862 31204 tablet_bootstrap.cc:654] T 6725de54cf924f959a51c59d3cd06af8 P 5361647f33364ecaa92a8acff75e9d1f: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:03.148072 31206 tablet_bootstrap.cc:492] T 6725de54cf924f959a51c59d3cd06af8 P ac12075ba1b44514881e1db1e42c4594: Bootstrap starting.
I20260811 05:17:03.148221 31204 log.cc:824] T 6725de54cf924f959a51c59d3cd06af8 P 5361647f33364ecaa92a8acff75e9d1f: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:03.148730 31206 tablet_bootstrap.cc:654] T 6725de54cf924f959a51c59d3cd06af8 P ac12075ba1b44514881e1db1e42c4594: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:03.149044 31210 tablet_bootstrap.cc:492] T 6725de54cf924f959a51c59d3cd06af8 P ee7547adbaa443409bcd9482a2891d55: Bootstrap starting.
I20260811 05:17:03.149099 31206 log.cc:824] T 6725de54cf924f959a51c59d3cd06af8 P ac12075ba1b44514881e1db1e42c4594: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:03.149129 31204 tablet_bootstrap.cc:492] T 6725de54cf924f959a51c59d3cd06af8 P 5361647f33364ecaa92a8acff75e9d1f: No bootstrap required, opened a new log
I20260811 05:17:03.149185 31204 ts_tablet_manager.cc:1397] T 6725de54cf924f959a51c59d3cd06af8 P 5361647f33364ecaa92a8acff75e9d1f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 05:17:03.149626 31210 tablet_bootstrap.cc:654] T 6725de54cf924f959a51c59d3cd06af8 P ee7547adbaa443409bcd9482a2891d55: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:03.149835 31206 tablet_bootstrap.cc:492] T 6725de54cf924f959a51c59d3cd06af8 P ac12075ba1b44514881e1db1e42c4594: No bootstrap required, opened a new log
I20260811 05:17:03.149914 31206 ts_tablet_manager.cc:1397] T 6725de54cf924f959a51c59d3cd06af8 P ac12075ba1b44514881e1db1e42c4594: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 05:17:03.149900 31210 log.cc:824] T 6725de54cf924f959a51c59d3cd06af8 P ee7547adbaa443409bcd9482a2891d55: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:03.150528 31210 tablet_bootstrap.cc:492] T 6725de54cf924f959a51c59d3cd06af8 P ee7547adbaa443409bcd9482a2891d55: No bootstrap required, opened a new log
I20260811 05:17:03.150588 31210 ts_tablet_manager.cc:1397] T 6725de54cf924f959a51c59d3cd06af8 P ee7547adbaa443409bcd9482a2891d55: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 05:17:03.151729 31204 raft_consensus.cc:348] T 6725de54cf924f959a51c59d3cd06af8 P 5361647f33364ecaa92a8acff75e9d1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac12075ba1b44514881e1db1e42c4594" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 32801 } } peers { permanent_uuid: "ee7547adbaa443409bcd9482a2891d55" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 33487 } } peers { permanent_uuid: "5361647f33364ecaa92a8acff75e9d1f" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37261 } }
I20260811 05:17:03.151913 31204 raft_consensus.cc:374] T 6725de54cf924f959a51c59d3cd06af8 P 5361647f33364ecaa92a8acff75e9d1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:03.151961 31204 raft_consensus.cc:729] T 6725de54cf924f959a51c59d3cd06af8 P 5361647f33364ecaa92a8acff75e9d1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5361647f33364ecaa92a8acff75e9d1f, State: Initialized, Role: FOLLOWER
I20260811 05:17:03.152122 31204 consensus_queue.cc:260] T 6725de54cf924f959a51c59d3cd06af8 P 5361647f33364ecaa92a8acff75e9d1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac12075ba1b44514881e1db1e42c4594" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 32801 } } peers { permanent_uuid: "ee7547adbaa443409bcd9482a2891d55" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 33487 } } peers { permanent_uuid: "5361647f33364ecaa92a8acff75e9d1f" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37261 } }
I20260811 05:17:03.152420 31204 ts_tablet_manager.cc:1428] T 6725de54cf924f959a51c59d3cd06af8 P 5361647f33364ecaa92a8acff75e9d1f: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260811 05:17:03.152460 30694 heartbeater.cc:503] Master 127.28.143.190:46411 was elected leader, sending a full tablet report...
I20260811 05:17:03.152369 31206 raft_consensus.cc:348] T 6725de54cf924f959a51c59d3cd06af8 P ac12075ba1b44514881e1db1e42c4594 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac12075ba1b44514881e1db1e42c4594" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 32801 } } peers { permanent_uuid: "ee7547adbaa443409bcd9482a2891d55" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 33487 } } peers { permanent_uuid: "5361647f33364ecaa92a8acff75e9d1f" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37261 } }
I20260811 05:17:03.152526 31206 raft_consensus.cc:374] T 6725de54cf924f959a51c59d3cd06af8 P ac12075ba1b44514881e1db1e42c4594 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:03.152557 31206 raft_consensus.cc:729] T 6725de54cf924f959a51c59d3cd06af8 P ac12075ba1b44514881e1db1e42c4594 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac12075ba1b44514881e1db1e42c4594, State: Initialized, Role: FOLLOWER
I20260811 05:17:03.152486 31210 raft_consensus.cc:348] T 6725de54cf924f959a51c59d3cd06af8 P ee7547adbaa443409bcd9482a2891d55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac12075ba1b44514881e1db1e42c4594" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 32801 } } peers { permanent_uuid: "ee7547adbaa443409bcd9482a2891d55" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 33487 } } peers { permanent_uuid: "5361647f33364ecaa92a8acff75e9d1f" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37261 } }
I20260811 05:17:03.152659 31210 raft_consensus.cc:374] T 6725de54cf924f959a51c59d3cd06af8 P ee7547adbaa443409bcd9482a2891d55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:03.152693 31210 raft_consensus.cc:729] T 6725de54cf924f959a51c59d3cd06af8 P ee7547adbaa443409bcd9482a2891d55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee7547adbaa443409bcd9482a2891d55, State: Initialized, Role: FOLLOWER
I20260811 05:17:03.152758 31206 consensus_queue.cc:260] T 6725de54cf924f959a51c59d3cd06af8 P ac12075ba1b44514881e1db1e42c4594 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac12075ba1b44514881e1db1e42c4594" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 32801 } } peers { permanent_uuid: "ee7547adbaa443409bcd9482a2891d55" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 33487 } } peers { permanent_uuid: "5361647f33364ecaa92a8acff75e9d1f" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37261 } }
I20260811 05:17:03.152813 31210 consensus_queue.cc:260] T 6725de54cf924f959a51c59d3cd06af8 P ee7547adbaa443409bcd9482a2891d55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac12075ba1b44514881e1db1e42c4594" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 32801 } } peers { permanent_uuid: "ee7547adbaa443409bcd9482a2891d55" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 33487 } } peers { permanent_uuid: "5361647f33364ecaa92a8acff75e9d1f" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37261 } }
I20260811 05:17:03.153071 31210 ts_tablet_manager.cc:1428] T 6725de54cf924f959a51c59d3cd06af8 P ee7547adbaa443409bcd9482a2891d55: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 05:17:03.153193 30934 heartbeater.cc:503] Master 127.28.143.190:46411 was elected leader, sending a full tablet report...
I20260811 05:17:03.153113 31206 ts_tablet_manager.cc:1428] T 6725de54cf924f959a51c59d3cd06af8 P ac12075ba1b44514881e1db1e42c4594: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 05:17:03.153519 31176 heartbeater.cc:503] Master 127.28.143.190:46411 was elected leader, sending a full tablet report...
W20260811 05:17:03.223959 30936 tablet.cc:2369] T 6725de54cf924f959a51c59d3cd06af8 P ee7547adbaa443409bcd9482a2891d55: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 05:17:03.323279 30696 tablet.cc:2369] T 6725de54cf924f959a51c59d3cd06af8 P 5361647f33364ecaa92a8acff75e9d1f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 05:17:03.360307 31179 tablet.cc:2369] T 6725de54cf924f959a51c59d3cd06af8 P ac12075ba1b44514881e1db1e42c4594: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:17:03.506731 31218 raft_consensus.cc:482] T 6725de54cf924f959a51c59d3cd06af8 P ac12075ba1b44514881e1db1e42c4594 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:17:03.506888 31218 raft_consensus.cc:504] T 6725de54cf924f959a51c59d3cd06af8 P ac12075ba1b44514881e1db1e42c4594 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac12075ba1b44514881e1db1e42c4594" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 32801 } } peers { permanent_uuid: "ee7547adbaa443409bcd9482a2891d55" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 33487 } } peers { permanent_uuid: "5361647f33364ecaa92a8acff75e9d1f" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37261 } }
I20260811 05:17:03.507241 31218 leader_election.cc:288] T 6725de54cf924f959a51c59d3cd06af8 P ac12075ba1b44514881e1db1e42c4594 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee7547adbaa443409bcd9482a2891d55 (127.28.143.130:33487), 5361647f33364ecaa92a8acff75e9d1f (127.28.143.129:37261)
I20260811 05:17:03.511219 31200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:03.507497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:48895 (local address 127.28.143.130:33487)
0811 05:17:03.507547 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:03.507549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:03.507562 (+ 13us) server_negotiation.cc:407] Connection header received
0811 05:17:03.507639 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:03.507640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:03.507653 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:03.507719 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:03.507896 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:03.508709 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:03.510243 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:03.510811 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:03.510899 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:03.510908 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:03.511094 (+ 186us) server_negotiation.cc:299] Negotiation successful
0811 05:17:03.511137 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 05:17:03.511219 31199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:03.507564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:43921 (local address 127.28.143.129:37261)
0811 05:17:03.507626 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:03.507628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:03.507908 (+ 280us) server_negotiation.cc:407] Connection header received
0811 05:17:03.507939 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:03.507940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:03.507952 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:03.508013 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:03.508155 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:03.508858 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:03.510242 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:03.510812 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:03.510899 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:03.510908 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:03.511095 (+ 187us) server_negotiation.cc:299] Negotiation successful
0811 05:17:03.511137 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 05:17:03.511231 31180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:03.507439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:33487 (local address 127.28.143.131:48895)
0811 05:17:03.507507 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:03.507520 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:03.507568 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:03.507735 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:03.507736 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:03.507737 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:03.507812 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:03.507815 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:03.508806 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:03.508808 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:03.510110 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:03.510114 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:03.510943 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:03.510944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:03.511009 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:03.511018 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 05:17:03.511078 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 05:17:03.511129 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":2}
I20260811 05:17:03.511241 31219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:03.507448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:37261 (local address 127.28.143.131:43921)
0811 05:17:03.507781 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:03.507794 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:03.507857 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:03.508034 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:03.508035 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:03.508036 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:03.508090 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:03.508091 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:03.508945 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:03.508946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:03.510110 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:03.510114 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:03.510918 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:03.510919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:03.511009 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:03.511018 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 05:17:03.511078 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 05:17:03.511127 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"mutex_wait_us":157,"thread_start_us":115,"threads_started":1}
I20260811 05:17:03.511606 30855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6725de54cf924f959a51c59d3cd06af8" candidate_uuid: "ac12075ba1b44514881e1db1e42c4594" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee7547adbaa443409bcd9482a2891d55" is_pre_election: true
I20260811 05:17:03.511615 30613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6725de54cf924f959a51c59d3cd06af8" candidate_uuid: "ac12075ba1b44514881e1db1e42c4594" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5361647f33364ecaa92a8acff75e9d1f" is_pre_election: true
I20260811 05:17:03.511782 30855 raft_consensus.cc:2454] T 6725de54cf924f959a51c59d3cd06af8 P ee7547adbaa443409bcd9482a2891d55 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac12075ba1b44514881e1db1e42c4594 in term 0.
I20260811 05:17:03.511790 30613 raft_consensus.cc:2454] T 6725de54cf924f959a51c59d3cd06af8 P 5361647f33364ecaa92a8acff75e9d1f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac12075ba1b44514881e1db1e42c4594 in term 0.
I20260811 05:17:03.512086 30968 leader_election.cc:302] T 6725de54cf924f959a51c59d3cd06af8 P ac12075ba1b44514881e1db1e42c4594 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac12075ba1b44514881e1db1e42c4594, ee7547adbaa443409bcd9482a2891d55; no voters:
I20260811 05:17:03.512266 31218 raft_consensus.cc:2781] T 6725de54cf924f959a51c59d3cd06af8 P ac12075ba1b44514881e1db1e42c4594 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:17:03.512342 31218 raft_consensus.cc:482] T 6725de54cf924f959a51c59d3cd06af8 P ac12075ba1b44514881e1db1e42c4594 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:17:03.512367 31218 raft_consensus.cc:3037] T 6725de54cf924f959a51c59d3cd06af8 P ac12075ba1b44514881e1db1e42c4594 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:03.513180 31218 raft_consensus.cc:504] T 6725de54cf924f959a51c59d3cd06af8 P ac12075ba1b44514881e1db1e42c4594 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac12075ba1b44514881e1db1e42c4594" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 32801 } } peers { permanent_uuid: "ee7547adbaa443409bcd9482a2891d55" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 33487 } } peers { permanent_uuid: "5361647f33364ecaa92a8acff75e9d1f" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37261 } }
I20260811 05:17:03.513334 31218 leader_election.cc:288] T 6725de54cf924f959a51c59d3cd06af8 P ac12075ba1b44514881e1db1e42c4594 [CANDIDATE]: Term 1 election: Requested vote from peers ee7547adbaa443409bcd9482a2891d55 (127.28.143.130:33487), 5361647f33364ecaa92a8acff75e9d1f (127.28.143.129:37261)
I20260811 05:17:03.513535 30855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6725de54cf924f959a51c59d3cd06af8" candidate_uuid: "ac12075ba1b44514881e1db1e42c4594" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee7547adbaa443409bcd9482a2891d55"
I20260811 05:17:03.513528 30613 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6725de54cf924f959a51c59d3cd06af8" candidate_uuid: "ac12075ba1b44514881e1db1e42c4594" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5361647f33364ecaa92a8acff75e9d1f"
I20260811 05:17:03.513615 30855 raft_consensus.cc:3037] T 6725de54cf924f959a51c59d3cd06af8 P ee7547adbaa443409bcd9482a2891d55 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:03.513615 30613 raft_consensus.cc:3037] T 6725de54cf924f959a51c59d3cd06af8 P 5361647f33364ecaa92a8acff75e9d1f [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:03.514353 30855 raft_consensus.cc:2454] T 6725de54cf924f959a51c59d3cd06af8 P ee7547adbaa443409bcd9482a2891d55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac12075ba1b44514881e1db1e42c4594 in term 1.
I20260811 05:17:03.514390 30613 raft_consensus.cc:2454] T 6725de54cf924f959a51c59d3cd06af8 P 5361647f33364ecaa92a8acff75e9d1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac12075ba1b44514881e1db1e42c4594 in term 1.
I20260811 05:17:03.514564 30966 leader_election.cc:302] T 6725de54cf924f959a51c59d3cd06af8 P ac12075ba1b44514881e1db1e42c4594 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5361647f33364ecaa92a8acff75e9d1f, ac12075ba1b44514881e1db1e42c4594; no voters:
I20260811 05:17:03.514678 31218 raft_consensus.cc:2781] T 6725de54cf924f959a51c59d3cd06af8 P ac12075ba1b44514881e1db1e42c4594 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:03.514835 31218 raft_consensus.cc:686] T 6725de54cf924f959a51c59d3cd06af8 P ac12075ba1b44514881e1db1e42c4594 [term 1 LEADER]: Becoming Leader. State: Replica: ac12075ba1b44514881e1db1e42c4594, State: Running, Role: LEADER
I20260811 05:17:03.514935 31218 consensus_queue.cc:237] T 6725de54cf924f959a51c59d3cd06af8 P ac12075ba1b44514881e1db1e42c4594 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac12075ba1b44514881e1db1e42c4594" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 32801 } } peers { permanent_uuid: "ee7547adbaa443409bcd9482a2891d55" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 33487 } } peers { permanent_uuid: "5361647f33364ecaa92a8acff75e9d1f" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37261 } }
I20260811 05:17:03.515870 30356 catalog_manager.cc:5495] T 6725de54cf924f959a51c59d3cd06af8 P ac12075ba1b44514881e1db1e42c4594 reported cstate change: term changed from 0 to 1, leader changed from <none> to ac12075ba1b44514881e1db1e42c4594 (127.28.143.131). New cstate: current_term: 1 leader_uuid: "ac12075ba1b44514881e1db1e42c4594" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac12075ba1b44514881e1db1e42c4594" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 32801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee7547adbaa443409bcd9482a2891d55" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 33487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5361647f33364ecaa92a8acff75e9d1f" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37261 } health_report { overall_health: UNKNOWN } } }
I20260811 05:17:03.591562 31202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:03.588975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37376 (local address 127.28.143.131:32801)
0811 05:17:03.589052 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:03.589054 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:03.589063 (+ 9us) server_negotiation.cc:407] Connection header received
0811 05:17:03.589100 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:03.589101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:03.589123 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:03.589191 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:17:03.589512 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:03.589995 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:03.590804 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:03.590984 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:03.591030 (+ 46us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:03.591192 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:17:03.591285 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:17:03.591355 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:03.591475 (+ 120us) server_negotiation.cc:299] Negotiation successful
0811 05:17:03.591483 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 05:17:03.593463 30613 raft_consensus.cc:1261] T 6725de54cf924f959a51c59d3cd06af8 P 5361647f33364ecaa92a8acff75e9d1f [term 1 FOLLOWER]: Refusing update from remote peer ac12075ba1b44514881e1db1e42c4594: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:17:03.593462 30855 raft_consensus.cc:1261] T 6725de54cf924f959a51c59d3cd06af8 P ee7547adbaa443409bcd9482a2891d55 [term 1 FOLLOWER]: Refusing update from remote peer ac12075ba1b44514881e1db1e42c4594: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:17:03.593818 31218 consensus_queue.cc:1035] T 6725de54cf924f959a51c59d3cd06af8 P ac12075ba1b44514881e1db1e42c4594 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5361647f33364ecaa92a8acff75e9d1f" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 37261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:17:03.593971 31222 consensus_queue.cc:1035] T 6725de54cf924f959a51c59d3cd06af8 P ac12075ba1b44514881e1db1e42c4594 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee7547adbaa443409bcd9482a2891d55" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 33487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:17:03.596450 31232 mvcc.cc:205] Tried to move back new op lower bound from 7317198535036096512 to 7317198534717730816. Current Snapshot: MvccSnapshot[applied={T|T < 7317198535036096512}]
I20260811 05:17:03.598776 31230 mvcc.cc:205] Tried to move back new op lower bound from 7317198535036096512 to 7317198534717730816. Current Snapshot: MvccSnapshot[applied={T|T < 7317198535036096512 or (T in {7317198535036096512})}]
I20260811 05:17:03.598944 30355 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:17:03.599040 30355 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:17:03.599725 30355 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 07ab75168df3490ea87c7bd15e14c8bd: Sending DeleteTablet for 3 replicas of tablet 6725de54cf924f959a51c59d3cd06af8
I20260811 05:17:03.600072 30579 tablet_service.cc:1514] Processing DeleteTablet for tablet 6725de54cf924f959a51c59d3cd06af8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:17:03 UTC) from {username='slave'} at 127.0.0.1:42702
I20260811 05:17:03.600076 30804 tablet_service.cc:1514] Processing DeleteTablet for tablet 6725de54cf924f959a51c59d3cd06af8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:17:03 UTC) from {username='slave'} at 127.0.0.1:40138
I20260811 05:17:03.600077 31069 tablet_service.cc:1514] Processing DeleteTablet for tablet 6725de54cf924f959a51c59d3cd06af8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:17:03 UTC) from {username='slave'} at 127.0.0.1:37360
I20260811 05:17:03.600320 31248 tablet_replica.cc:330] T 6725de54cf924f959a51c59d3cd06af8 P ac12075ba1b44514881e1db1e42c4594: stopping tablet replica
I20260811 05:17:03.600417 31248 raft_consensus.cc:2229] T 6725de54cf924f959a51c59d3cd06af8 P ac12075ba1b44514881e1db1e42c4594 [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:17:03.600515 31248 raft_consensus.cc:2258] T 6725de54cf924f959a51c59d3cd06af8 P ac12075ba1b44514881e1db1e42c4594 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:17:03.600687 31249 tablet_replica.cc:330] T 6725de54cf924f959a51c59d3cd06af8 P 5361647f33364ecaa92a8acff75e9d1f: stopping tablet replica
I20260811 05:17:03.600710 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 30462
I20260811 05:17:03.600777 31249 raft_consensus.cc:2229] T 6725de54cf924f959a51c59d3cd06af8 P 5361647f33364ecaa92a8acff75e9d1f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:17:03.600898 31249 raft_consensus.cc:2258] T 6725de54cf924f959a51c59d3cd06af8 P 5361647f33364ecaa92a8acff75e9d1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:17:03.601374 31248 ts_tablet_manager.cc:1905] T 6725de54cf924f959a51c59d3cd06af8 P ac12075ba1b44514881e1db1e42c4594: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:17:03.603583 31248 ts_tablet_manager.cc:1918] T 6725de54cf924f959a51c59d3cd06af8 P ac12075ba1b44514881e1db1e42c4594: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:17:03.603669 31248 log.cc:1196] T 6725de54cf924f959a51c59d3cd06af8 P ac12075ba1b44514881e1db1e42c4594: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-2/wal/wals/6725de54cf924f959a51c59d3cd06af8
I20260811 05:17:03.603938 31248 ts_tablet_manager.cc:1939] T 6725de54cf924f959a51c59d3cd06af8 P ac12075ba1b44514881e1db1e42c4594: Deleting consensus metadata
I20260811 05:17:03.604067 31250 tablet_replica.cc:330] T 6725de54cf924f959a51c59d3cd06af8 P ee7547adbaa443409bcd9482a2891d55: stopping tablet replica
I20260811 05:17:03.604231 31250 raft_consensus.cc:2229] T 6725de54cf924f959a51c59d3cd06af8 P ee7547adbaa443409bcd9482a2891d55 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:17:03.604393 31250 raft_consensus.cc:2258] T 6725de54cf924f959a51c59d3cd06af8 P ee7547adbaa443409bcd9482a2891d55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:17:03.604591 30330 catalog_manager.cc:4841] TS ac12075ba1b44514881e1db1e42c4594 (127.28.143.131:32801): tablet 6725de54cf924f959a51c59d3cd06af8 (table test-table [id=7bd6ac0d14c0496b86982caf90749a5d]) successfully deleted
I20260811 05:17:03.608976 31250 ts_tablet_manager.cc:1905] T 6725de54cf924f959a51c59d3cd06af8 P ee7547adbaa443409bcd9482a2891d55: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:17:03.612733 31250 ts_tablet_manager.cc:1918] T 6725de54cf924f959a51c59d3cd06af8 P ee7547adbaa443409bcd9482a2891d55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:17:03.612838 31250 log.cc:1196] T 6725de54cf924f959a51c59d3cd06af8 P ee7547adbaa443409bcd9482a2891d55: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425159060450-29246-0/minicluster-data/ts-1/wal/wals/6725de54cf924f959a51c59d3cd06af8
I20260811 05:17:03.613090 31250 ts_tablet_manager.cc:1939] T 6725de54cf924f959a51c59d3cd06af8 P ee7547adbaa443409bcd9482a2891d55: Deleting consensus metadata
I20260811 05:17:03.613552 30326 catalog_manager.cc:4841] TS ee7547adbaa443409bcd9482a2891d55 (127.28.143.130:33487): tablet 6725de54cf924f959a51c59d3cd06af8 (table test-table [id=7bd6ac0d14c0496b86982caf90749a5d]) successfully deleted
W20260811 05:17:03.627200 30325 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.143.129:37261 (error 108)
I20260811 05:17:03.627319 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 30698
I20260811 05:17:03.645979 31196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:03.627691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:37261 (local address 127.0.0.1:42714)
0811 05:17:03.645791 (+ 18100us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:03.645908 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.129:37261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15989}
W20260811 05:17:03.646214 30325 catalog_manager.cc:4584] TS 5361647f33364ecaa92a8acff75e9d1f (127.28.143.129:37261): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6725de54cf924f959a51c59d3cd06af8: Network error: Client connection negotiation failed: client connection to 127.28.143.129:37261: connect: Connection refused (error 111)
I20260811 05:17:03.663524 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 30940
I20260811 05:17:03.678282 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 30296
2026-08-11T05:17:03Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1179 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:17:03 dist-test-slave-f7th krb5kdc[31256](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:17:03 dist-test-slave-f7th krb5kdc[31256](info): set up 2 sockets
Aug 11 05:17:03 dist-test-slave-f7th krb5kdc[31256](info): commencing operation
krb5kdc: starting...
W20260811 05:17:04.910290 29246 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Aug 11 05:17:05 dist-test-slave-f7th krb5kdc[31256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425425, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:17:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:17:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
I20260811 05:17:05.051288 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:37365
--webserver_interface=127.28.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:36803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:37365
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:05.138060 31282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:05.138289 31282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:05.138322 31282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:05.139501 31282 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:05.139547 31282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:05.139565 31282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:05.139582 31282 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:05.139597 31282 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:05.141188 31282 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:36803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:37365
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:37365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31282
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:17:05.141470 31282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:05.141693 31282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:05.147044 31291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:05.146993 31287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:05.147006 31288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:05.147583 31282 server_base.cc:1034] running on GCE node
I20260811 05:17:05.147866 31282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:05.148216 31282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:05.149370 31282 hybrid_clock.cc:648] HybridClock initialized: now 1786425425149350 us; error 34 us; skew 500 ppm
Aug 11 05:17:05 dist-test-slave-f7th krb5kdc[31256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425425, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:05.151930 31282 init.cc:377] Logged in from keytab as kudu/127.28.143.190@KRBTEST.COM (short username kudu)
I20260811 05:17:05.152416 31282 webserver.cc:460] Webserver started at http://127.28.143.190:38597/ using document root <none> and password file <none>
I20260811 05:17:05.152673 31282 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:05.152734 31282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:05.152838 31282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:05.153919 31282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "2b7e4cad6740444b9ba9e5493b023cbf"
format_stamp: "Formatted at 2026-08-11 05:17:05 on dist-test-slave-f7th"
server_key: "ea97e8b3f56dc5ce65acdc59616bb684"
server_key_iv: "23264889513b505789011ee721ba97fa"
server_key_version: "encryptionkey@0"
I20260811 05:17:05.154240 31282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "2b7e4cad6740444b9ba9e5493b023cbf"
format_stamp: "Formatted at 2026-08-11 05:17:05 on dist-test-slave-f7th"
server_key: "ea97e8b3f56dc5ce65acdc59616bb684"
server_key_iv: "23264889513b505789011ee721ba97fa"
server_key_version: "encryptionkey@0"
I20260811 05:17:05.155624 31282 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 05:17:05.156481 31311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:05.156801 31282 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:17:05.156925 31282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "2b7e4cad6740444b9ba9e5493b023cbf"
format_stamp: "Formatted at 2026-08-11 05:17:05 on dist-test-slave-f7th"
server_key: "ea97e8b3f56dc5ce65acdc59616bb684"
server_key_iv: "23264889513b505789011ee721ba97fa"
server_key_version: "encryptionkey@0"
I20260811 05:17:05.157001 31282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:05.169528 31282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:05.171672 31282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:05.171844 31282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:05.176047 31282 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:37365
I20260811 05:17:05.176095 31415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:37365 every 8 connection(s)
I20260811 05:17:05.176460 31282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:17:05.177413 31417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:05.177898 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 31282
I20260811 05:17:05.178013 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:17:05.178200 29246 external_mini_cluster.cc:1428] Setting key c0bdc299df47efe44f86f6734b419cae
I20260811 05:17:05.180531 31417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b7e4cad6740444b9ba9e5493b023cbf: Bootstrap starting.
I20260811 05:17:05.181202 31417 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b7e4cad6740444b9ba9e5493b023cbf: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:05.181505 31417 log.cc:824] T 00000000000000000000000000000000 P 2b7e4cad6740444b9ba9e5493b023cbf: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:05.182230 31417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b7e4cad6740444b9ba9e5493b023cbf: No bootstrap required, opened a new log
I20260811 05:17:05.184538 31417 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b7e4cad6740444b9ba9e5493b023cbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b7e4cad6740444b9ba9e5493b023cbf" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 37365 } }
I20260811 05:17:05.184739 31417 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b7e4cad6740444b9ba9e5493b023cbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:05.184784 31417 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b7e4cad6740444b9ba9e5493b023cbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b7e4cad6740444b9ba9e5493b023cbf, State: Initialized, Role: FOLLOWER
I20260811 05:17:05.184927 31417 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b7e4cad6740444b9ba9e5493b023cbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b7e4cad6740444b9ba9e5493b023cbf" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 37365 } }
I20260811 05:17:05.185009 31417 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b7e4cad6740444b9ba9e5493b023cbf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:17:05.185053 31417 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b7e4cad6740444b9ba9e5493b023cbf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:17:05.185103 31417 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b7e4cad6740444b9ba9e5493b023cbf [term 0 FOLLOWER]: Advancing to term 1
Aug 11 05:17:05 dist-test-slave-f7th krb5kdc[31256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425425, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:17:05.185739 31417 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b7e4cad6740444b9ba9e5493b023cbf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b7e4cad6740444b9ba9e5493b023cbf" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 37365 } }
I20260811 05:17:05.185858 31417 leader_election.cc:302] T 00000000000000000000000000000000 P 2b7e4cad6740444b9ba9e5493b023cbf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b7e4cad6740444b9ba9e5493b023cbf; no voters:
I20260811 05:17:05.186049 31417 leader_election.cc:288] T 00000000000000000000000000000000 P 2b7e4cad6740444b9ba9e5493b023cbf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:17:05.186156 31425 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b7e4cad6740444b9ba9e5493b023cbf [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:05.186357 31417 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b7e4cad6740444b9ba9e5493b023cbf [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:05.186369 31425 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b7e4cad6740444b9ba9e5493b023cbf [term 1 LEADER]: Becoming Leader. State: Replica: 2b7e4cad6740444b9ba9e5493b023cbf, State: Running, Role: LEADER
I20260811 05:17:05.186590 31425 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b7e4cad6740444b9ba9e5493b023cbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b7e4cad6740444b9ba9e5493b023cbf" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 37365 } }
I20260811 05:17:05.187003 31427 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b7e4cad6740444b9ba9e5493b023cbf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b7e4cad6740444b9ba9e5493b023cbf. Latest consensus state: current_term: 1 leader_uuid: "2b7e4cad6740444b9ba9e5493b023cbf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b7e4cad6740444b9ba9e5493b023cbf" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 37365 } } }
I20260811 05:17:05.187005 31426 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b7e4cad6740444b9ba9e5493b023cbf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b7e4cad6740444b9ba9e5493b023cbf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b7e4cad6740444b9ba9e5493b023cbf" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 37365 } } }
I20260811 05:17:05.187115 31427 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b7e4cad6740444b9ba9e5493b023cbf [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:05.187121 31426 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b7e4cad6740444b9ba9e5493b023cbf [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:05.187352 31442 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:17:05.187891 31442 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:17:05.189563 31442 catalog_manager.cc:1347] Generated new cluster ID: 66048cd2644d4471a137e19292843ce4
I20260811 05:17:05.189618 31442 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:17:05.189826 31423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:05.178775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34326 (local address 127.28.143.190:37365)
0811 05:17:05.178970 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:05.178980 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:05.179015 (+ 35us) server_negotiation.cc:407] Connection header received
0811 05:17:05.179471 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:05.179479 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:05.179531 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:05.179657 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:05.180512 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:05.181566 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:05.182378 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:05.183014 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:05.183147 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:05.185622 (+ 2475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:05.185627 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:05.185628 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:05.185641 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:05.188028 (+ 2387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:05.188440 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:05.188443 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:05.188443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:05.188550 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:05.188818 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:05.188818 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:05.188819 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:05.189137 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:05.189288 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:05.189422 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 05:17:05.189571 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":88,"threads_started":1}
I20260811 05:17:05.208513 31442 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:17:05.209110 31442 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:17:05.214702 31442 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b7e4cad6740444b9ba9e5493b023cbf: Generated new TSK 0
I20260811 05:17:05.214972 31442 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.143.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab.
Entry for principal kudu/127.28.143.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab.
I20260811 05:17:05.252930 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.129:0
--local_ip_for_outbound_sockets=127.28.143.129
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:37365
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:36803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:05.339601 31468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:05.339834 31468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:05.339869 31468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:05.341121 31468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:05.341171 31468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:05.341209 31468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.129
I20260811 05:17:05.342777 31468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:36803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.129.keytab
--principal=kudu/127.28.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:37365
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.129
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:17:05.343047 31468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:05.343292 31468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:05.348486 31491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:05.348480 31487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:05.348480 31486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:05.348857 31468 server_base.cc:1034] running on GCE node
I20260811 05:17:05.349087 31468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:05.349327 31468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:05.350507 31468 hybrid_clock.cc:648] HybridClock initialized: now 1786425425350491 us; error 34 us; skew 500 ppm
Aug 11 05:17:05 dist-test-slave-f7th krb5kdc[31256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425425, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:05.353266 31468 init.cc:377] Logged in from keytab as kudu/127.28.143.129@KRBTEST.COM (short username kudu)
I20260811 05:17:05.353771 31468 webserver.cc:460] Webserver started at http://127.28.143.129:44291/ using document root <none> and password file <none>
I20260811 05:17:05.353963 31468 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:05.354010 31468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:05.354115 31468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:05.355131 31468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-0/data/instance:
uuid: "0e811f4ca43f409eb0d7b0a30202917e"
format_stamp: "Formatted at 2026-08-11 05:17:05 on dist-test-slave-f7th"
server_key: "b7bb0540e3cdb6fe7eb4c8227c13592e"
server_key_iv: "4ab66be24d2d80d172828865406d024e"
server_key_version: "encryptionkey@0"
I20260811 05:17:05.355430 31468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance:
uuid: "0e811f4ca43f409eb0d7b0a30202917e"
format_stamp: "Formatted at 2026-08-11 05:17:05 on dist-test-slave-f7th"
server_key: "b7bb0540e3cdb6fe7eb4c8227c13592e"
server_key_iv: "4ab66be24d2d80d172828865406d024e"
server_key_version: "encryptionkey@0"
I20260811 05:17:05.356746 31468 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 05:17:05.357611 31503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:05.357787 31468 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:17:05.357880 31468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-0/wal
uuid: "0e811f4ca43f409eb0d7b0a30202917e"
format_stamp: "Formatted at 2026-08-11 05:17:05 on dist-test-slave-f7th"
server_key: "b7bb0540e3cdb6fe7eb4c8227c13592e"
server_key_iv: "4ab66be24d2d80d172828865406d024e"
server_key_version: "encryptionkey@0"
I20260811 05:17:05.357939 31468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:05.375293 31468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:05.377723 31468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:05.377856 31468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:05.378099 31468 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:17:05.378400 31468 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:17:05.378433 31468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:05.378456 31468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:17:05.378469 31468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:05.384131 31468 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.129:43485
I20260811 05:17:05.384197 31714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.129:43485 every 8 connection(s)
I20260811 05:17:05.384531 31468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
I20260811 05:17:05.390199 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 31468
I20260811 05:17:05.390347 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance
I20260811 05:17:05.390585 29246 external_mini_cluster.cc:1428] Setting key 9d912f6ac9e79cd4549ee20856397304
Aug 11 05:17:05 dist-test-slave-f7th krb5kdc[31256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425425, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.129@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:17:05.394850 31423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:05.385425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:47639 (local address 127.28.143.190:37365)
0811 05:17:05.385474 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:05.385476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:05.385717 (+ 241us) server_negotiation.cc:407] Connection header received
0811 05:17:05.386259 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:05.386260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:05.386273 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:05.386327 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:05.387342 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:05.387850 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:05.388866 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:05.389081 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:05.389138 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:05.391591 (+ 2453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:05.391594 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:05.391594 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:05.391603 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:05.393176 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:05.393751 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:05.393753 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:05.393753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:05.393809 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:05.394147 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:05.394148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:05.394149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:05.394279 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:05.394345 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:05.394681 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 05:17:05.394767 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 05:17:05.395107 31721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:05.385374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:37365 (local address 127.28.143.129:47639)
0811 05:17:05.385588 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:05.385627 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:05.386154 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:05.386399 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:05.386403 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:05.387009 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:05.387222 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:05.387227 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:05.387937 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:05.387938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:05.388789 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:05.388792 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:05.389153 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:05.389154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:05.389276 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:05.389739 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:17:05.389747 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:05.391475 (+ 1728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:05.393286 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:05.393298 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:05.393301 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:05.393673 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:05.393866 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:05.393867 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:05.393867 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:05.394019 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:05.394347 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:05.394354 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:05.394538 (+ 184us) client_negotiation.cc:769] Sending connection context
0811 05:17:05.394668 (+ 130us) client_negotiation.cc:240] Negotiation successful
0811 05:17:05.394845 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":115,"threads_started":1}
I20260811 05:17:05.395746 31719 heartbeater.cc:347] Connected to a master server at 127.28.143.190:37365
I20260811 05:17:05.395866 31719 heartbeater.cc:464] Registering TS with master...
I20260811 05:17:05.396165 31719 heartbeater.cc:511] Master 127.28.143.190:37365 requested a full tablet report, sending...
I20260811 05:17:05.396855 31340 ts_manager.cc:194] Registered new tserver with Master: 0e811f4ca43f409eb0d7b0a30202917e (127.28.143.129:43485)
I20260811 05:17:05.397541 31340 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.129@KRBTEST.COM'} at 127.28.143.129:47639
WARNING: no policy specified for kudu/127.28.143.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab.
Entry for principal kudu/127.28.143.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab.
I20260811 05:17:05.425272 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.130:0
--local_ip_for_outbound_sockets=127.28.143.130
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:37365
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:36803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:05.513041 31729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:05.513255 31729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:05.513288 31729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:05.514426 31729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:05.514472 31729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:05.514508 31729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.130
I20260811 05:17:05.515983 31729 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:36803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.130.keytab
--principal=kudu/127.28.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:37365
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.130
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:17:05.516253 31729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:05.516470 31729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:05.521565 31737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:05.521574 31742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:05.521637 31739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:05.521641 31729 server_base.cc:1034] running on GCE node
I20260811 05:17:05.522045 31729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:05.522269 31729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:05.523428 31729 hybrid_clock.cc:648] HybridClock initialized: now 1786425425523412 us; error 28 us; skew 500 ppm
Aug 11 05:17:05 dist-test-slave-f7th krb5kdc[31256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425425, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:05.525965 31729 init.cc:377] Logged in from keytab as kudu/127.28.143.130@KRBTEST.COM (short username kudu)
I20260811 05:17:05.526419 31729 webserver.cc:460] Webserver started at http://127.28.143.130:44533/ using document root <none> and password file <none>
I20260811 05:17:05.526621 31729 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:05.526670 31729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:05.526803 31729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:05.527858 31729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-1/data/instance:
uuid: "1bd2caefb57b4846b2af49e6e74c7158"
format_stamp: "Formatted at 2026-08-11 05:17:05 on dist-test-slave-f7th"
server_key: "46b065198fffa85df3680863eb99302a"
server_key_iv: "5547c29e8fb922881518dd016589c01f"
server_key_version: "encryptionkey@0"
I20260811 05:17:05.528196 31729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance:
uuid: "1bd2caefb57b4846b2af49e6e74c7158"
format_stamp: "Formatted at 2026-08-11 05:17:05 on dist-test-slave-f7th"
server_key: "46b065198fffa85df3680863eb99302a"
server_key_iv: "5547c29e8fb922881518dd016589c01f"
server_key_version: "encryptionkey@0"
I20260811 05:17:05.529626 31729 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 05:17:05.530454 31754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:05.530682 31729 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:17:05.530782 31729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-1/wal
uuid: "1bd2caefb57b4846b2af49e6e74c7158"
format_stamp: "Formatted at 2026-08-11 05:17:05 on dist-test-slave-f7th"
server_key: "46b065198fffa85df3680863eb99302a"
server_key_iv: "5547c29e8fb922881518dd016589c01f"
server_key_version: "encryptionkey@0"
I20260811 05:17:05.530857 31729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:05.544574 31729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:05.546978 31729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:05.547147 31729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:05.547405 31729 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:17:05.547731 31729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:17:05.547780 31729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:05.547811 31729 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:17:05.547828 31729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:05.553985 31729 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.130:37961
I20260811 05:17:05.554044 31938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.130:37961 every 8 connection(s)
I20260811 05:17:05.554383 31729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
Aug 11 05:17:05 dist-test-slave-f7th krb5kdc[31256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425425, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.130@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:17:05.562438 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 31729
I20260811 05:17:05.562563 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance
I20260811 05:17:05.562740 29246 external_mini_cluster.cc:1428] Setting key 6c9a4f33a5d58277d9422249c1b31a00
I20260811 05:17:05.563813 31423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:05.555114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:39099 (local address 127.28.143.190:37365)
0811 05:17:05.555193 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:05.555195 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:05.555356 (+ 161us) server_negotiation.cc:407] Connection header received
0811 05:17:05.555858 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:05.555860 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:05.555870 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:05.555970 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:05.557059 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:05.557567 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:05.558567 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:05.558765 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:05.558820 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:05.561110 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:05.561112 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:05.561112 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:05.561120 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:05.562382 (+ 1262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:05.562810 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:05.562811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:05.562811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:05.562867 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:05.563165 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:05.563165 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:05.563166 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:05.563277 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:05.563337 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:05.563642 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 05:17:05.563729 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 05:17:05.563921 31944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:05.555074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:37365 (local address 127.28.143.130:39099)
0811 05:17:05.555236 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:05.555275 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:05.555752 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:05.556073 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:05.556077 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:05.556746 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:05.556954 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:05.556960 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:05.557656 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:05.557657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:05.558482 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:05.558484 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:05.558828 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:05.558828 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:05.558931 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:05.559444 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:17:05.559471 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:05.561015 (+ 1544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:05.562464 (+ 1449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:05.562471 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:05.562473 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:05.562742 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:05.562966 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:05.562967 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:05.562968 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:05.563090 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:05.563335 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:05.563340 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:05.563508 (+ 168us) client_negotiation.cc:769] Sending connection context
0811 05:17:05.563601 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 05:17:05.563737 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":81,"threads_started":1}
I20260811 05:17:05.565642 31940 heartbeater.cc:347] Connected to a master server at 127.28.143.190:37365
I20260811 05:17:05.565757 31940 heartbeater.cc:464] Registering TS with master...
I20260811 05:17:05.565976 31940 heartbeater.cc:511] Master 127.28.143.190:37365 requested a full tablet report, sending...
I20260811 05:17:05.566377 31340 ts_manager.cc:194] Registered new tserver with Master: 1bd2caefb57b4846b2af49e6e74c7158 (127.28.143.130:37961)
I20260811 05:17:05.566869 31340 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.130@KRBTEST.COM'} at 127.28.143.130:39099
WARNING: no policy specified for kudu/127.28.143.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab.
Entry for principal kudu/127.28.143.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab.
I20260811 05:17:05.592542 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.131:0
--local_ip_for_outbound_sockets=127.28.143.131
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:37365
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:36803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:05.682209 31947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:05.682451 31947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:05.682487 31947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:05.683718 31947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:05.683769 31947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:05.683818 31947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.131
I20260811 05:17:05.685400 31947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:36803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.131.keytab
--principal=kudu/127.28.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:37365
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.131
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:17:05.685664 31947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:05.685890 31947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 05:17:05.691103 31947 server_base.cc:1034] running on GCE node
W20260811 05:17:05.691074 31959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:05.691092 31961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:05.691187 31963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:05.691597 31947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:05.691838 31947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:05.692979 31947 hybrid_clock.cc:648] HybridClock initialized: now 1786425425692962 us; error 29 us; skew 500 ppm
Aug 11 05:17:05 dist-test-slave-f7th krb5kdc[31256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425425, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:05.695573 31947 init.cc:377] Logged in from keytab as kudu/127.28.143.131@KRBTEST.COM (short username kudu)
I20260811 05:17:05.696053 31947 webserver.cc:460] Webserver started at http://127.28.143.131:36085/ using document root <none> and password file <none>
I20260811 05:17:05.696272 31947 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:05.696323 31947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:05.696455 31947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:05.697593 31947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-2/data/instance:
uuid: "a22079f85c8c478ba460a421fb86536b"
format_stamp: "Formatted at 2026-08-11 05:17:05 on dist-test-slave-f7th"
server_key: "9e33cb0a7193b6603ecdd9faaf08b32a"
server_key_iv: "c282e8688a7a7363b78bf67dab59ede0"
server_key_version: "encryptionkey@0"
I20260811 05:17:05.697945 31947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance:
uuid: "a22079f85c8c478ba460a421fb86536b"
format_stamp: "Formatted at 2026-08-11 05:17:05 on dist-test-slave-f7th"
server_key: "9e33cb0a7193b6603ecdd9faaf08b32a"
server_key_iv: "c282e8688a7a7363b78bf67dab59ede0"
server_key_version: "encryptionkey@0"
I20260811 05:17:05.699353 31947 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 05:17:05.700189 31975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:05.700343 31947 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:17:05.700433 31947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-2/wal
uuid: "a22079f85c8c478ba460a421fb86536b"
format_stamp: "Formatted at 2026-08-11 05:17:05 on dist-test-slave-f7th"
server_key: "9e33cb0a7193b6603ecdd9faaf08b32a"
server_key_iv: "c282e8688a7a7363b78bf67dab59ede0"
server_key_version: "encryptionkey@0"
I20260811 05:17:05.700496 31947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:05.732407 31947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:05.734925 31947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:05.735083 31947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:05.735335 31947 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:17:05.735670 31947 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:17:05.735705 31947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:05.735736 31947 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:17:05.735760 31947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:05.741451 31947 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.131:46293
I20260811 05:17:05.741549 32190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.131:46293 every 8 connection(s)
I20260811 05:17:05.741842 31947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
Aug 11 05:17:05 dist-test-slave-f7th krb5kdc[31256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425425, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.131@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:17:05.749661 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 31947
I20260811 05:17:05.749754 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance
I20260811 05:17:05.749923 29246 external_mini_cluster.cc:1428] Setting key b419e1205bb99c4a14e7f3d085229900
I20260811 05:17:05.751396 31423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:05.742613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:49015 (local address 127.28.143.190:37365)
0811 05:17:05.742662 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:05.742665 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:05.742866 (+ 201us) server_negotiation.cc:407] Connection header received
0811 05:17:05.743342 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:05.743343 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:05.743355 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:05.743408 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:05.744382 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:05.744944 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:05.746039 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:05.746233 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:05.746298 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:05.748589 (+ 2291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:05.748590 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:05.748591 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:05.748598 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:05.750022 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:05.750493 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:05.750495 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:05.750495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:05.750562 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:05.750800 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:05.750800 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:05.750801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:05.750947 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:05.750999 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:05.751249 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 05:17:05.751319 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 05:17:05.751561 32197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:05.742596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:37365 (local address 127.28.143.131:49015)
0811 05:17:05.742747 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:05.742782 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:05.743248 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:05.743480 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:05.743484 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:05.744070 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:05.744273 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:05.744277 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:05.745115 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:05.745116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:05.745934 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:05.745936 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:05.746306 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:05.746307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:05.746407 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:05.746914 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:17:05.746929 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:05.748472 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:05.750102 (+ 1630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:05.750109 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:05.750111 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:05.750418 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:05.750624 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:05.750625 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:05.750625 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:05.750736 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:05.751000 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:05.751005 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:05.751130 (+ 125us) client_negotiation.cc:769] Sending connection context
0811 05:17:05.751219 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 05:17:05.751350 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":86,"threads_started":1}
I20260811 05:17:05.752027 32192 heartbeater.cc:347] Connected to a master server at 127.28.143.190:37365
I20260811 05:17:05.752117 32192 heartbeater.cc:464] Registering TS with master...
I20260811 05:17:05.752353 32192 heartbeater.cc:511] Master 127.28.143.190:37365 requested a full tablet report, sending...
I20260811 05:17:05.752784 31340 ts_manager.cc:194] Registered new tserver with Master: a22079f85c8c478ba460a421fb86536b (127.28.143.131:46293)
I20260811 05:17:05.753188 31340 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.143.131@KRBTEST.COM'} at 127.28.143.131:49015
I20260811 05:17:05.762086 29246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:17:05.770502 31423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:05.763602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34340 (local address 127.28.143.190:37365)
0811 05:17:05.763687 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:05.763688 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:05.763853 (+ 165us) server_negotiation.cc:407] Connection header received
0811 05:17:05.763882 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:05.763883 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:05.763896 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:05.763959 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:05.764703 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:05.765446 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:05.766482 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:05.766715 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:05.766776 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:05.768184 (+ 1408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:05.768185 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:05.768186 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:05.768193 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:05.769555 (+ 1362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:05.769867 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:05.769867 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:05.769868 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:05.769916 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:05.770062 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:05.770063 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:05.770063 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:05.770209 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:05.770282 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:05.770355 (+ 73us) server_negotiation.cc:299] Negotiation successful
0811 05:17:05.770421 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 05:17:05.772089 31340 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34340:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:17:05.773772 31340 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:17:05.781608 32230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:05.777308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:37961 (local address 127.0.0.1:54474)
0811 05:17:05.777865 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:05.777876 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:05.777919 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:05.778357 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:05.778357 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:05.778359 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:05.778438 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:05.778440 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:05.779281 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:05.779282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:05.780404 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:05.780407 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:05.781305 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:05.781306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:05.781388 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:05.781408 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 05:17:05.781481 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 05:17:05.781533 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"mutex_wait_us":74,"thread_start_us":158,"threads_started":1}
I20260811 05:17:05.781651 32233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:05.778008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54474 (local address 127.28.143.130:37961)
0811 05:17:05.778222 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:05.778226 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:05.778236 (+ 10us) server_negotiation.cc:407] Connection header received
0811 05:17:05.778272 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:05.778274 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:05.778290 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:05.778362 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:05.778497 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:05.779187 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:05.780498 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:05.781196 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:05.781279 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:05.781349 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:05.781510 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 05:17:05.781556 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":95,"threads_started":1}
I20260811 05:17:05.783263 32218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:05.777079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:43485 (local address 127.0.0.1:41848)
0811 05:17:05.777264 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:05.777310 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:05.777386 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:05.778191 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:05.778192 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:05.778194 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:05.778233 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:05.778234 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:05.779482 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:05.779483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:05.780866 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:05.780869 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:05.783013 (+ 2144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:05.783014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:05.783066 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:05.783070 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:17:05.783118 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 05:17:05.783169 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":43,"thread_start_us":118,"threads_started":1}
I20260811 05:17:05.783293 32224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:05.777231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41848 (local address 127.28.143.129:43485)
0811 05:17:05.778052 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:05.778056 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:05.778063 (+ 7us) server_negotiation.cc:407] Connection header received
0811 05:17:05.778093 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:05.778094 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:05.778108 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:05.778184 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:05.778294 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:05.779377 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:05.780970 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:05.782137 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:05.782224 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:05.782303 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:05.783162 (+ 859us) server_negotiation.cc:299] Negotiation successful
0811 05:17:05.783218 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":786,"thread_start_us":99,"threads_started":1}
I20260811 05:17:05.782953 31831 tablet_service.cc:1467] Processing CreateTablet for tablet 23fc30bcc0c5482f979af7f274c9a433 (DEFAULT_TABLE table=test-table [id=122ba30266c94a79a4aa4a5ced00b13e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:17:05.783684 31831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23fc30bcc0c5482f979af7f274c9a433. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:05.783738 32221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:05.777074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.131:46293 (local address 127.0.0.1:55504)
0811 05:17:05.777261 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:05.777309 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:05.777436 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:05.777959 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:05.777962 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:05.777965 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:05.778054 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:05.778057 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:05.779467 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:05.779469 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:05.780865 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:05.780869 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:05.782974 (+ 2105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:05.782975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:05.783184 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:05.783194 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 05:17:05.783271 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 05:17:05.783648 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":15,"thread_start_us":81,"threads_started":1}
I20260811 05:17:05.783978 31591 tablet_service.cc:1467] Processing CreateTablet for tablet 23fc30bcc0c5482f979af7f274c9a433 (DEFAULT_TABLE table=test-table [id=122ba30266c94a79a4aa4a5ced00b13e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:17:05.784087 32231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:05.777555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55504 (local address 127.28.143.131:46293)
0811 05:17:05.777728 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:05.777734 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:05.777745 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:17:05.777790 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:05.777792 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:05.777810 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:05.777899 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:05.778144 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:05.779320 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:05.782349 (+ 3029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:05.782866 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:05.782933 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:05.782967 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:05.783936 (+ 969us) server_negotiation.cc:299] Negotiation successful
0811 05:17:05.783996 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":88,"threads_started":1}
I20260811 05:17:05.784322 31591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23fc30bcc0c5482f979af7f274c9a433. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:05.784803 32064 tablet_service.cc:1467] Processing CreateTablet for tablet 23fc30bcc0c5482f979af7f274c9a433 (DEFAULT_TABLE table=test-table [id=122ba30266c94a79a4aa4a5ced00b13e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:17:05.785168 32064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23fc30bcc0c5482f979af7f274c9a433. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:05.786862 32234 tablet_bootstrap.cc:492] T 23fc30bcc0c5482f979af7f274c9a433 P 1bd2caefb57b4846b2af49e6e74c7158: Bootstrap starting.
I20260811 05:17:05.787510 32234 tablet_bootstrap.cc:654] T 23fc30bcc0c5482f979af7f274c9a433 P 1bd2caefb57b4846b2af49e6e74c7158: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:05.787859 32235 tablet_bootstrap.cc:492] T 23fc30bcc0c5482f979af7f274c9a433 P 0e811f4ca43f409eb0d7b0a30202917e: Bootstrap starting.
I20260811 05:17:05.788049 32234 log.cc:824] T 23fc30bcc0c5482f979af7f274c9a433 P 1bd2caefb57b4846b2af49e6e74c7158: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:05.788511 32235 tablet_bootstrap.cc:654] T 23fc30bcc0c5482f979af7f274c9a433 P 0e811f4ca43f409eb0d7b0a30202917e: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:05.788826 32240 tablet_bootstrap.cc:492] T 23fc30bcc0c5482f979af7f274c9a433 P a22079f85c8c478ba460a421fb86536b: Bootstrap starting.
I20260811 05:17:05.789031 32234 tablet_bootstrap.cc:492] T 23fc30bcc0c5482f979af7f274c9a433 P 1bd2caefb57b4846b2af49e6e74c7158: No bootstrap required, opened a new log
I20260811 05:17:05.789072 32235 log.cc:824] T 23fc30bcc0c5482f979af7f274c9a433 P 0e811f4ca43f409eb0d7b0a30202917e: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:05.789110 32234 ts_tablet_manager.cc:1397] T 23fc30bcc0c5482f979af7f274c9a433 P 1bd2caefb57b4846b2af49e6e74c7158: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 05:17:05.789469 32240 tablet_bootstrap.cc:654] T 23fc30bcc0c5482f979af7f274c9a433 P a22079f85c8c478ba460a421fb86536b: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:05.789935 32235 tablet_bootstrap.cc:492] T 23fc30bcc0c5482f979af7f274c9a433 P 0e811f4ca43f409eb0d7b0a30202917e: No bootstrap required, opened a new log
I20260811 05:17:05.789970 32240 log.cc:824] T 23fc30bcc0c5482f979af7f274c9a433 P a22079f85c8c478ba460a421fb86536b: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:05.790011 32235 ts_tablet_manager.cc:1397] T 23fc30bcc0c5482f979af7f274c9a433 P 0e811f4ca43f409eb0d7b0a30202917e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260811 05:17:05.790786 32240 tablet_bootstrap.cc:492] T 23fc30bcc0c5482f979af7f274c9a433 P a22079f85c8c478ba460a421fb86536b: No bootstrap required, opened a new log
I20260811 05:17:05.790866 32240 ts_tablet_manager.cc:1397] T 23fc30bcc0c5482f979af7f274c9a433 P a22079f85c8c478ba460a421fb86536b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 05:17:05.791080 32234 raft_consensus.cc:348] T 23fc30bcc0c5482f979af7f274c9a433 P 1bd2caefb57b4846b2af49e6e74c7158 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e811f4ca43f409eb0d7b0a30202917e" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43485 } } peers { permanent_uuid: "1bd2caefb57b4846b2af49e6e74c7158" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37961 } } peers { permanent_uuid: "a22079f85c8c478ba460a421fb86536b" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 46293 } }
I20260811 05:17:05.791286 32234 raft_consensus.cc:374] T 23fc30bcc0c5482f979af7f274c9a433 P 1bd2caefb57b4846b2af49e6e74c7158 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:05.791337 32234 raft_consensus.cc:729] T 23fc30bcc0c5482f979af7f274c9a433 P 1bd2caefb57b4846b2af49e6e74c7158 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bd2caefb57b4846b2af49e6e74c7158, State: Initialized, Role: FOLLOWER
I20260811 05:17:05.791463 32234 consensus_queue.cc:260] T 23fc30bcc0c5482f979af7f274c9a433 P 1bd2caefb57b4846b2af49e6e74c7158 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e811f4ca43f409eb0d7b0a30202917e" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43485 } } peers { permanent_uuid: "1bd2caefb57b4846b2af49e6e74c7158" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37961 } } peers { permanent_uuid: "a22079f85c8c478ba460a421fb86536b" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 46293 } }
I20260811 05:17:05.791808 31940 heartbeater.cc:503] Master 127.28.143.190:37365 was elected leader, sending a full tablet report...
I20260811 05:17:05.791790 32234 ts_tablet_manager.cc:1428] T 23fc30bcc0c5482f979af7f274c9a433 P 1bd2caefb57b4846b2af49e6e74c7158: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 05:17:05.792013 32235 raft_consensus.cc:348] T 23fc30bcc0c5482f979af7f274c9a433 P 0e811f4ca43f409eb0d7b0a30202917e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e811f4ca43f409eb0d7b0a30202917e" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43485 } } peers { permanent_uuid: "1bd2caefb57b4846b2af49e6e74c7158" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37961 } } peers { permanent_uuid: "a22079f85c8c478ba460a421fb86536b" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 46293 } }
I20260811 05:17:05.792187 32235 raft_consensus.cc:374] T 23fc30bcc0c5482f979af7f274c9a433 P 0e811f4ca43f409eb0d7b0a30202917e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:05.792225 32235 raft_consensus.cc:729] T 23fc30bcc0c5482f979af7f274c9a433 P 0e811f4ca43f409eb0d7b0a30202917e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e811f4ca43f409eb0d7b0a30202917e, State: Initialized, Role: FOLLOWER
I20260811 05:17:05.792352 32235 consensus_queue.cc:260] T 23fc30bcc0c5482f979af7f274c9a433 P 0e811f4ca43f409eb0d7b0a30202917e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e811f4ca43f409eb0d7b0a30202917e" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43485 } } peers { permanent_uuid: "1bd2caefb57b4846b2af49e6e74c7158" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37961 } } peers { permanent_uuid: "a22079f85c8c478ba460a421fb86536b" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 46293 } }
I20260811 05:17:05.792692 32235 ts_tablet_manager.cc:1428] T 23fc30bcc0c5482f979af7f274c9a433 P 0e811f4ca43f409eb0d7b0a30202917e: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 05:17:05.792748 31719 heartbeater.cc:503] Master 127.28.143.190:37365 was elected leader, sending a full tablet report...
I20260811 05:17:05.792711 32240 raft_consensus.cc:348] T 23fc30bcc0c5482f979af7f274c9a433 P a22079f85c8c478ba460a421fb86536b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e811f4ca43f409eb0d7b0a30202917e" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43485 } } peers { permanent_uuid: "1bd2caefb57b4846b2af49e6e74c7158" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37961 } } peers { permanent_uuid: "a22079f85c8c478ba460a421fb86536b" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 46293 } }
I20260811 05:17:05.792888 32240 raft_consensus.cc:374] T 23fc30bcc0c5482f979af7f274c9a433 P a22079f85c8c478ba460a421fb86536b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:05.792963 32240 raft_consensus.cc:729] T 23fc30bcc0c5482f979af7f274c9a433 P a22079f85c8c478ba460a421fb86536b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a22079f85c8c478ba460a421fb86536b, State: Initialized, Role: FOLLOWER
I20260811 05:17:05.793083 32240 consensus_queue.cc:260] T 23fc30bcc0c5482f979af7f274c9a433 P a22079f85c8c478ba460a421fb86536b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e811f4ca43f409eb0d7b0a30202917e" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43485 } } peers { permanent_uuid: "1bd2caefb57b4846b2af49e6e74c7158" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37961 } } peers { permanent_uuid: "a22079f85c8c478ba460a421fb86536b" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 46293 } }
I20260811 05:17:05.793360 32240 ts_tablet_manager.cc:1428] T 23fc30bcc0c5482f979af7f274c9a433 P a22079f85c8c478ba460a421fb86536b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 05:17:05.793414 32192 heartbeater.cc:503] Master 127.28.143.190:37365 was elected leader, sending a full tablet report...
I20260811 05:17:05.799801 32245 raft_consensus.cc:482] T 23fc30bcc0c5482f979af7f274c9a433 P 0e811f4ca43f409eb0d7b0a30202917e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:17:05.800081 32245 raft_consensus.cc:504] T 23fc30bcc0c5482f979af7f274c9a433 P 0e811f4ca43f409eb0d7b0a30202917e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e811f4ca43f409eb0d7b0a30202917e" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43485 } } peers { permanent_uuid: "1bd2caefb57b4846b2af49e6e74c7158" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37961 } } peers { permanent_uuid: "a22079f85c8c478ba460a421fb86536b" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 46293 } }
I20260811 05:17:05.800535 32245 leader_election.cc:288] T 23fc30bcc0c5482f979af7f274c9a433 P 0e811f4ca43f409eb0d7b0a30202917e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1bd2caefb57b4846b2af49e6e74c7158 (127.28.143.130:37961), a22079f85c8c478ba460a421fb86536b (127.28.143.131:46293)
I20260811 05:17:05.803875 32233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:05.800856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:35355 (local address 127.28.143.130:37961)
0811 05:17:05.800918 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:05.800920 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:05.800931 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:17:05.801034 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:05.801035 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:05.801046 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:05.801132 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:05.801369 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:05.801937 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:05.802962 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:05.803482 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:05.803546 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:05.803556 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:05.803745 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 05:17:05.803790 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 05:17:05.803875 32231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:05.800864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:37679 (local address 127.28.143.131:46293)
0811 05:17:05.800918 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:05.800920 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:05.801231 (+ 311us) server_negotiation.cc:407] Connection header received
0811 05:17:05.801266 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:05.801267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:05.801280 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:05.801352 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:05.801505 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:05.802057 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:05.803110 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:05.803509 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:05.803583 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:05.803594 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:05.803744 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 05:17:05.803789 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 05:17:05.803893 31721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:05.800753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:37961 (local address 127.28.143.129:35355)
0811 05:17:05.800829 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:05.800842 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:05.800888 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:05.801195 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:05.801197 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:05.801200 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:05.801291 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:05.801293 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:05.802018 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:05.802020 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:05.802869 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:05.802872 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:05.803566 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:05.803567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:05.803669 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:05.803674 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 05:17:05.803727 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 05:17:05.803809 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":3}
I20260811 05:17:05.803908 32247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:05.800753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.131:46293 (local address 127.28.143.129:37679)
0811 05:17:05.801112 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:05.801124 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:05.801199 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:05.801361 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:05.801362 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:05.801363 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:05.801441 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:05.801442 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:05.802142 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:05.802143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:05.803008 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:05.803011 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:05.803599 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:05.803601 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:05.803671 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:05.803675 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:17:05.803727 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 05:17:05.803801 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"mutex_wait_us":156,"thread_start_us":129,"threads_started":1}
I20260811 05:17:05.804297 31863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23fc30bcc0c5482f979af7f274c9a433" candidate_uuid: "0e811f4ca43f409eb0d7b0a30202917e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bd2caefb57b4846b2af49e6e74c7158" is_pre_election: true
I20260811 05:17:05.804297 32108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23fc30bcc0c5482f979af7f274c9a433" candidate_uuid: "0e811f4ca43f409eb0d7b0a30202917e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a22079f85c8c478ba460a421fb86536b" is_pre_election: true
I20260811 05:17:05.804499 32108 raft_consensus.cc:2454] T 23fc30bcc0c5482f979af7f274c9a433 P a22079f85c8c478ba460a421fb86536b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e811f4ca43f409eb0d7b0a30202917e in term 0.
I20260811 05:17:05.804502 31863 raft_consensus.cc:2454] T 23fc30bcc0c5482f979af7f274c9a433 P 1bd2caefb57b4846b2af49e6e74c7158 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e811f4ca43f409eb0d7b0a30202917e in term 0.
I20260811 05:17:05.804811 31504 leader_election.cc:302] T 23fc30bcc0c5482f979af7f274c9a433 P 0e811f4ca43f409eb0d7b0a30202917e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e811f4ca43f409eb0d7b0a30202917e, a22079f85c8c478ba460a421fb86536b; no voters:
W20260811 05:17:05.804888 31941 tablet.cc:2369] T 23fc30bcc0c5482f979af7f274c9a433 P 1bd2caefb57b4846b2af49e6e74c7158: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:17:05.805179 32245 raft_consensus.cc:2781] T 23fc30bcc0c5482f979af7f274c9a433 P 0e811f4ca43f409eb0d7b0a30202917e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:17:05.805246 32245 raft_consensus.cc:482] T 23fc30bcc0c5482f979af7f274c9a433 P 0e811f4ca43f409eb0d7b0a30202917e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:17:05.805269 32245 raft_consensus.cc:3037] T 23fc30bcc0c5482f979af7f274c9a433 P 0e811f4ca43f409eb0d7b0a30202917e [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:05.806039 32245 raft_consensus.cc:504] T 23fc30bcc0c5482f979af7f274c9a433 P 0e811f4ca43f409eb0d7b0a30202917e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e811f4ca43f409eb0d7b0a30202917e" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43485 } } peers { permanent_uuid: "1bd2caefb57b4846b2af49e6e74c7158" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37961 } } peers { permanent_uuid: "a22079f85c8c478ba460a421fb86536b" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 46293 } }
I20260811 05:17:05.806200 32245 leader_election.cc:288] T 23fc30bcc0c5482f979af7f274c9a433 P 0e811f4ca43f409eb0d7b0a30202917e [CANDIDATE]: Term 1 election: Requested vote from peers 1bd2caefb57b4846b2af49e6e74c7158 (127.28.143.130:37961), a22079f85c8c478ba460a421fb86536b (127.28.143.131:46293)
I20260811 05:17:05.806473 31863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23fc30bcc0c5482f979af7f274c9a433" candidate_uuid: "0e811f4ca43f409eb0d7b0a30202917e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bd2caefb57b4846b2af49e6e74c7158"
I20260811 05:17:05.806540 32108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23fc30bcc0c5482f979af7f274c9a433" candidate_uuid: "0e811f4ca43f409eb0d7b0a30202917e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a22079f85c8c478ba460a421fb86536b"
I20260811 05:17:05.806581 31863 raft_consensus.cc:3037] T 23fc30bcc0c5482f979af7f274c9a433 P 1bd2caefb57b4846b2af49e6e74c7158 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:05.806643 32108 raft_consensus.cc:3037] T 23fc30bcc0c5482f979af7f274c9a433 P a22079f85c8c478ba460a421fb86536b [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:05.807446 32108 raft_consensus.cc:2454] T 23fc30bcc0c5482f979af7f274c9a433 P a22079f85c8c478ba460a421fb86536b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e811f4ca43f409eb0d7b0a30202917e in term 1.
I20260811 05:17:05.807449 31863 raft_consensus.cc:2454] T 23fc30bcc0c5482f979af7f274c9a433 P 1bd2caefb57b4846b2af49e6e74c7158 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e811f4ca43f409eb0d7b0a30202917e in term 1.
I20260811 05:17:05.807636 31504 leader_election.cc:302] T 23fc30bcc0c5482f979af7f274c9a433 P 0e811f4ca43f409eb0d7b0a30202917e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e811f4ca43f409eb0d7b0a30202917e, a22079f85c8c478ba460a421fb86536b; no voters:
I20260811 05:17:05.807735 32245 raft_consensus.cc:2781] T 23fc30bcc0c5482f979af7f274c9a433 P 0e811f4ca43f409eb0d7b0a30202917e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:05.807916 32245 raft_consensus.cc:686] T 23fc30bcc0c5482f979af7f274c9a433 P 0e811f4ca43f409eb0d7b0a30202917e [term 1 LEADER]: Becoming Leader. State: Replica: 0e811f4ca43f409eb0d7b0a30202917e, State: Running, Role: LEADER
I20260811 05:17:05.808017 32245 consensus_queue.cc:237] T 23fc30bcc0c5482f979af7f274c9a433 P 0e811f4ca43f409eb0d7b0a30202917e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e811f4ca43f409eb0d7b0a30202917e" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43485 } } peers { permanent_uuid: "1bd2caefb57b4846b2af49e6e74c7158" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37961 } } peers { permanent_uuid: "a22079f85c8c478ba460a421fb86536b" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 46293 } }
I20260811 05:17:05.809001 31340 catalog_manager.cc:5495] T 23fc30bcc0c5482f979af7f274c9a433 P 0e811f4ca43f409eb0d7b0a30202917e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e811f4ca43f409eb0d7b0a30202917e (127.28.143.129). New cstate: current_term: 1 leader_uuid: "0e811f4ca43f409eb0d7b0a30202917e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e811f4ca43f409eb0d7b0a30202917e" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 43485 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1bd2caefb57b4846b2af49e6e74c7158" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a22079f85c8c478ba460a421fb86536b" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 46293 } health_report { overall_health: UNKNOWN } } }
I20260811 05:17:05.819490 32224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:05.817216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41864 (local address 127.28.143.129:43485)
0811 05:17:05.817273 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:05.817274 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:05.817282 (+ 8us) server_negotiation.cc:407] Connection header received
0811 05:17:05.817320 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:05.817321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:05.817341 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:05.817402 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:17:05.817528 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:05.818019 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:05.818811 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:05.819014 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:05.819110 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:05.819217 (+ 107us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:17:05.819326 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:17:05.819385 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:05.819398 (+ 13us) server_negotiation.cc:299] Negotiation successful
0811 05:17:05.819418 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 05:17:05.821652 31863 raft_consensus.cc:1261] T 23fc30bcc0c5482f979af7f274c9a433 P 1bd2caefb57b4846b2af49e6e74c7158 [term 1 FOLLOWER]: Refusing update from remote peer 0e811f4ca43f409eb0d7b0a30202917e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:17:05.821677 32108 raft_consensus.cc:1261] T 23fc30bcc0c5482f979af7f274c9a433 P a22079f85c8c478ba460a421fb86536b [term 1 FOLLOWER]: Refusing update from remote peer 0e811f4ca43f409eb0d7b0a30202917e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:17:05.822057 32248 consensus_queue.cc:1035] T 23fc30bcc0c5482f979af7f274c9a433 P 0e811f4ca43f409eb0d7b0a30202917e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bd2caefb57b4846b2af49e6e74c7158" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 37961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:17:05.822196 32245 consensus_queue.cc:1035] T 23fc30bcc0c5482f979af7f274c9a433 P 0e811f4ca43f409eb0d7b0a30202917e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a22079f85c8c478ba460a421fb86536b" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 46293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:17:05.825287 32253 mvcc.cc:205] Tried to move back new op lower bound from 7317198544162254848 to 7317198544110391296. Current Snapshot: MvccSnapshot[applied={T|T < 7317198544162254848}]
I20260811 05:17:05.827638 31340 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34340:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:17:05.827742 31340 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34340:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:17:05.828464 31340 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2b7e4cad6740444b9ba9e5493b023cbf: Sending DeleteTablet for 3 replicas of tablet 23fc30bcc0c5482f979af7f274c9a433
I20260811 05:17:05.828845 32064 tablet_service.cc:1514] Processing DeleteTablet for tablet 23fc30bcc0c5482f979af7f274c9a433 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:17:05 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:55504
I20260811 05:17:05.828925 31831 tablet_service.cc:1514] Processing DeleteTablet for tablet 23fc30bcc0c5482f979af7f274c9a433 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:17:05 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:54474
I20260811 05:17:05.829141 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 31468
I20260811 05:17:05.829255 31591 tablet_service.cc:1514] Processing DeleteTablet for tablet 23fc30bcc0c5482f979af7f274c9a433 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:17:05 UTC) from {username='kudu', principal='kudu/127.28.143.190@KRBTEST.COM'} at 127.0.0.1:41848
I20260811 05:17:05.833119 32267 tablet_replica.cc:330] T 23fc30bcc0c5482f979af7f274c9a433 P a22079f85c8c478ba460a421fb86536b: stopping tablet replica
I20260811 05:17:05.833336 32267 raft_consensus.cc:2229] T 23fc30bcc0c5482f979af7f274c9a433 P a22079f85c8c478ba460a421fb86536b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:17:05.832964 32268 tablet_replica.cc:330] T 23fc30bcc0c5482f979af7f274c9a433 P 1bd2caefb57b4846b2af49e6e74c7158: stopping tablet replica
I20260811 05:17:05.833613 32268 raft_consensus.cc:2229] T 23fc30bcc0c5482f979af7f274c9a433 P 1bd2caefb57b4846b2af49e6e74c7158 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:17:05.833851 32267 raft_consensus.cc:2258] T 23fc30bcc0c5482f979af7f274c9a433 P a22079f85c8c478ba460a421fb86536b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:17:05.834131 32268 raft_consensus.cc:2258] T 23fc30bcc0c5482f979af7f274c9a433 P 1bd2caefb57b4846b2af49e6e74c7158 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:17:05.836853 32268 ts_tablet_manager.cc:1905] T 23fc30bcc0c5482f979af7f274c9a433 P 1bd2caefb57b4846b2af49e6e74c7158: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:17:05.836839 32267 ts_tablet_manager.cc:1905] T 23fc30bcc0c5482f979af7f274c9a433 P a22079f85c8c478ba460a421fb86536b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:17:05.838826 32267 ts_tablet_manager.cc:1918] T 23fc30bcc0c5482f979af7f274c9a433 P a22079f85c8c478ba460a421fb86536b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:17:05.838861 32268 ts_tablet_manager.cc:1918] T 23fc30bcc0c5482f979af7f274c9a433 P 1bd2caefb57b4846b2af49e6e74c7158: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:17:05.839056 32268 log.cc:1196] T 23fc30bcc0c5482f979af7f274c9a433 P 1bd2caefb57b4846b2af49e6e74c7158: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-1/wal/wals/23fc30bcc0c5482f979af7f274c9a433
I20260811 05:17:05.839069 32267 log.cc:1196] T 23fc30bcc0c5482f979af7f274c9a433 P a22079f85c8c478ba460a421fb86536b: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425159060450-29246-0/minicluster-data/ts-2/wal/wals/23fc30bcc0c5482f979af7f274c9a433
I20260811 05:17:05.839463 32268 ts_tablet_manager.cc:1939] T 23fc30bcc0c5482f979af7f274c9a433 P 1bd2caefb57b4846b2af49e6e74c7158: Deleting consensus metadata
I20260811 05:17:05.839491 32267 ts_tablet_manager.cc:1939] T 23fc30bcc0c5482f979af7f274c9a433 P a22079f85c8c478ba460a421fb86536b: Deleting consensus metadata
I20260811 05:17:05.840052 31316 catalog_manager.cc:4841] TS 1bd2caefb57b4846b2af49e6e74c7158 (127.28.143.130:37961): tablet 23fc30bcc0c5482f979af7f274c9a433 (table test-table [id=122ba30266c94a79a4aa4a5ced00b13e]) successfully deleted
I20260811 05:17:05.840204 31313 catalog_manager.cc:4841] TS a22079f85c8c478ba460a421fb86536b (127.28.143.131:46293): tablet 23fc30bcc0c5482f979af7f274c9a433 (table test-table [id=122ba30266c94a79a4aa4a5ced00b13e]) successfully deleted
W20260811 05:17:05.844345 31316 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.143.129:43485 (error 108)
I20260811 05:17:05.844576 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 31729
I20260811 05:17:05.848049 32221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:05.847874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:43485 (local address 127.0.0.1:41878)
0811 05:17:05.847943 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:05.848002 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.129:43485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260811 05:17:05.848336 31316 catalog_manager.cc:4584] TS 0e811f4ca43f409eb0d7b0a30202917e (127.28.143.129:43485): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 23fc30bcc0c5482f979af7f274c9a433: Network error: Client connection negotiation failed: client connection to 127.28.143.129:43485: connect: Connection refused (error 111)
I20260811 05:17:05.862867 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 31947
I20260811 05:17:05.889827 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 31282
2026-08-11T05:17:05Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2216 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T05:17:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:17:05Z Disabled control of system clock
I20260811 05:17:05.931010 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:38073
--webserver_interface=127.28.143.190
--webserver_port=0
--builtin_ntp_servers=127.28.143.148:38121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:38073
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 05:17:06.017508 32285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:06.017735 32285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:06.017756 32285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:06.018939 32285 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:06.018994 32285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:06.019007 32285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:06.019018 32285 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:06.019029 32285 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:06.020478 32285 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:38121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:38073
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:38073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:17:06.020776 32285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:06.021107 32285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 05:17:06.026626 32285 server_base.cc:1034] running on GCE node
W20260811 05:17:06.026587 32293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:06.026603 32294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:06.026607 32298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:06.027356 32285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:06.027676 32285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:06.028831 32285 hybrid_clock.cc:648] HybridClock initialized: now 1786425426028806 us; error 39 us; skew 500 ppm
I20260811 05:17:06.030413 32285 webserver.cc:460] Webserver started at http://127.28.143.190:33005/ using document root <none> and password file <none>
I20260811 05:17:06.030673 32285 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:06.030726 32285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:06.030861 32285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:06.032003 32285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "bfc45aed75d3411ba310e80e532a2ffc"
format_stamp: "Formatted at 2026-08-11 05:17:06 on dist-test-slave-f7th"
server_key: "a4ab8adff35e54a306c022bd3fea5814"
server_key_iv: "301e34f8b87d835d73a0fd4ef7bce0cd"
server_key_version: "encryptionkey@0"
I20260811 05:17:06.032369 32285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "bfc45aed75d3411ba310e80e532a2ffc"
format_stamp: "Formatted at 2026-08-11 05:17:06 on dist-test-slave-f7th"
server_key: "a4ab8adff35e54a306c022bd3fea5814"
server_key_iv: "301e34f8b87d835d73a0fd4ef7bce0cd"
server_key_version: "encryptionkey@0"
I20260811 05:17:06.033816 32285 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 05:17:06.034677 32309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:06.034845 32285 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:17:06.034943 32285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "bfc45aed75d3411ba310e80e532a2ffc"
format_stamp: "Formatted at 2026-08-11 05:17:06 on dist-test-slave-f7th"
server_key: "a4ab8adff35e54a306c022bd3fea5814"
server_key_iv: "301e34f8b87d835d73a0fd4ef7bce0cd"
server_key_version: "encryptionkey@0"
I20260811 05:17:06.035018 32285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:06.052695 32285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:06.053061 32285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:06.053205 32285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:06.057314 32285 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:38073
I20260811 05:17:06.057364 32410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:38073 every 8 connection(s)
I20260811 05:17:06.057715 32285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:17:06.058707 32412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:06.061085 32412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfc45aed75d3411ba310e80e532a2ffc: Bootstrap starting.
I20260811 05:17:06.061627 32412 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bfc45aed75d3411ba310e80e532a2ffc: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:06.061868 32412 log.cc:824] T 00000000000000000000000000000000 P bfc45aed75d3411ba310e80e532a2ffc: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:06.062520 32412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfc45aed75d3411ba310e80e532a2ffc: No bootstrap required, opened a new log
I20260811 05:17:06.064122 32412 raft_consensus.cc:348] T 00000000000000000000000000000000 P bfc45aed75d3411ba310e80e532a2ffc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfc45aed75d3411ba310e80e532a2ffc" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 38073 } }
I20260811 05:17:06.064265 32412 raft_consensus.cc:374] T 00000000000000000000000000000000 P bfc45aed75d3411ba310e80e532a2ffc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:06.064285 32412 raft_consensus.cc:729] T 00000000000000000000000000000000 P bfc45aed75d3411ba310e80e532a2ffc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfc45aed75d3411ba310e80e532a2ffc, State: Initialized, Role: FOLLOWER
I20260811 05:17:06.064402 32412 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfc45aed75d3411ba310e80e532a2ffc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfc45aed75d3411ba310e80e532a2ffc" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 38073 } }
I20260811 05:17:06.064483 32412 raft_consensus.cc:388] T 00000000000000000000000000000000 P bfc45aed75d3411ba310e80e532a2ffc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:17:06.064523 32412 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfc45aed75d3411ba310e80e532a2ffc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:17:06.064563 32412 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfc45aed75d3411ba310e80e532a2ffc [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:06.065122 32412 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfc45aed75d3411ba310e80e532a2ffc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfc45aed75d3411ba310e80e532a2ffc" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 38073 } }
I20260811 05:17:06.065231 32412 leader_election.cc:302] T 00000000000000000000000000000000 P bfc45aed75d3411ba310e80e532a2ffc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bfc45aed75d3411ba310e80e532a2ffc; no voters:
I20260811 05:17:06.065379 32412 leader_election.cc:288] T 00000000000000000000000000000000 P bfc45aed75d3411ba310e80e532a2ffc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:17:06.065469 32415 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bfc45aed75d3411ba310e80e532a2ffc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:06.065637 32412 sys_catalog.cc:564] T 00000000000000000000000000000000 P bfc45aed75d3411ba310e80e532a2ffc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:06.065675 32415 raft_consensus.cc:686] T 00000000000000000000000000000000 P bfc45aed75d3411ba310e80e532a2ffc [term 1 LEADER]: Becoming Leader. State: Replica: bfc45aed75d3411ba310e80e532a2ffc, State: Running, Role: LEADER
I20260811 05:17:06.065799 32415 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfc45aed75d3411ba310e80e532a2ffc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfc45aed75d3411ba310e80e532a2ffc" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 38073 } }
I20260811 05:17:06.066287 32416 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfc45aed75d3411ba310e80e532a2ffc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bfc45aed75d3411ba310e80e532a2ffc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfc45aed75d3411ba310e80e532a2ffc" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 38073 } } }
I20260811 05:17:06.066361 32420 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfc45aed75d3411ba310e80e532a2ffc [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfc45aed75d3411ba310e80e532a2ffc. Latest consensus state: current_term: 1 leader_uuid: "bfc45aed75d3411ba310e80e532a2ffc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfc45aed75d3411ba310e80e532a2ffc" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 38073 } } }
I20260811 05:17:06.066471 32420 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfc45aed75d3411ba310e80e532a2ffc [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:06.066382 32416 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfc45aed75d3411ba310e80e532a2ffc [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:06.066931 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 32285
I20260811 05:17:06.066962 32435 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:17:06.067037 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:17:06.067256 29246 external_mini_cluster.cc:1428] Setting key 8e81a0f5d9747e892cea089715c0723e
I20260811 05:17:06.067551 32435 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:17:06.070883 32435 catalog_manager.cc:1347] Generated new cluster ID: 3b2b28733e0b4bb48e28ae686ae9aaa8
I20260811 05:17:06.070957 32435 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:17:06.073401 32451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:06.067759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41768 (local address 127.28.143.190:38073)
0811 05:17:06.068286 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:06.068296 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:06.068334 (+ 38us) server_negotiation.cc:407] Connection header received
0811 05:17:06.068504 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:06.068511 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:06.068582 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:06.068898 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:06.069010 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:06.070388 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:06.071364 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:06.071843 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:06.071963 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:06.072653 (+ 690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:06.072657 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:06.072657 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:06.072840 (+ 183us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:06.072874 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:06.072886 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:06.072944 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:06.073032 (+ 88us) server_negotiation.cc:299] Negotiation successful
0811 05:17:06.073206 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":97,"threads_started":1}
I20260811 05:17:06.084677 32435 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:17:06.085243 32435 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:17:06.098130 32435 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bfc45aed75d3411ba310e80e532a2ffc: Generated new TSK 0
I20260811 05:17:06.098346 32435 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:17:06.102664 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.129:0
--local_ip_for_outbound_sockets=127.28.143.129
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:38073
--builtin_ntp_servers=127.28.143.148:38121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 05:17:06.216661 32453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:06.216910 32453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:06.216935 32453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:06.218210 32453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:06.218272 32453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:06.218305 32453 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.129
I20260811 05:17:06.220019 32453 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:38121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.143.129
--webserver_port=0
--tserver_master_addrs=127.28.143.190:38073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.129
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:17:06.220290 32453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:06.220660 32453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:06.226905 32463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:06.226976 32467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:06.226930 32453 server_base.cc:1034] running on GCE node
W20260811 05:17:06.227231 32470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:06.227491 32453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:06.227759 32453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:06.228915 32453 hybrid_clock.cc:648] HybridClock initialized: now 1786425426228898 us; error 33 us; skew 500 ppm
I20260811 05:17:06.230263 32453 webserver.cc:460] Webserver started at http://127.28.143.129:34335/ using document root <none> and password file <none>
I20260811 05:17:06.230549 32453 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:06.230607 32453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:06.230760 32453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:06.231887 32453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-0/data/instance:
uuid: "3217a94ca9414c8ea135d593030a6ad2"
format_stamp: "Formatted at 2026-08-11 05:17:06 on dist-test-slave-f7th"
server_key: "3dba2f3c2093cbcc18213bd013df301e"
server_key_iv: "d130b751f5675d5c9405b037b2d64035"
server_key_version: "encryptionkey@0"
I20260811 05:17:06.232251 32453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance:
uuid: "3217a94ca9414c8ea135d593030a6ad2"
format_stamp: "Formatted at 2026-08-11 05:17:06 on dist-test-slave-f7th"
server_key: "3dba2f3c2093cbcc18213bd013df301e"
server_key_iv: "d130b751f5675d5c9405b037b2d64035"
server_key_version: "encryptionkey@0"
I20260811 05:17:06.233847 32453 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 05:17:06.234912 32478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:06.235219 32453 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:17:06.235420 32453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-0/wal
uuid: "3217a94ca9414c8ea135d593030a6ad2"
format_stamp: "Formatted at 2026-08-11 05:17:06 on dist-test-slave-f7th"
server_key: "3dba2f3c2093cbcc18213bd013df301e"
server_key_iv: "d130b751f5675d5c9405b037b2d64035"
server_key_version: "encryptionkey@0"
I20260811 05:17:06.235519 32453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:06.246163 32453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:06.246505 32453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:06.246649 32453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:06.246922 32453 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:17:06.247282 32453 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:17:06.247318 32453 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:06.247349 32453 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:17:06.247372 32453 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:06.253319 32453 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.129:41539
I20260811 05:17:06.253363 32680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.129:41539 every 8 connection(s)
I20260811 05:17:06.253744 32453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-0/data/info.pb
I20260811 05:17:06.259421 32451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:06.254536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:60659 (local address 127.28.143.190:38073)
0811 05:17:06.254627 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:06.254629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:06.254871 (+ 242us) server_negotiation.cc:407] Connection header received
0811 05:17:06.255458 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:06.255459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:06.255475 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:06.255530 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:06.256256 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:06.256815 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:06.257856 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:06.258058 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:06.258117 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:06.259124 (+ 1007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:06.259125 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:06.259125 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:06.259137 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:06.259153 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:06.259157 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:06.259203 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:06.259295 (+ 92us) server_negotiation.cc:299] Negotiation successful
0811 05:17:06.259331 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 05:17:06.259653 32684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:06.254513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:38073 (local address 127.28.143.129:60659)
0811 05:17:06.254738 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:06.254783 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:06.255336 (+ 553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:06.255605 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:06.255609 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:06.255618 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:06.256162 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:06.256168 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:06.256920 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:06.256921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:06.257749 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:06.257752 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:06.258145 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:06.258145 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:06.258283 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:06.258893 (+ 610us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:17:06.258894 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:06.258968 (+ 74us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:17:06.258999 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:06.259213 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:06.259220 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:06.259224 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:17:06.259288 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 05:17:06.259414 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":85,"threads_started":1}
I20260811 05:17:06.260294 32681 heartbeater.cc:347] Connected to a master server at 127.28.143.190:38073
I20260811 05:17:06.260397 32681 heartbeater.cc:464] Registering TS with master...
I20260811 05:17:06.260648 32681 heartbeater.cc:511] Master 127.28.143.190:38073 requested a full tablet report, sending...
I20260811 05:17:06.260716 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 32453
I20260811 05:17:06.260833 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-0/wal/instance
I20260811 05:17:06.261050 29246 external_mini_cluster.cc:1428] Setting key 179005160ab9e1e6320b11fa39f51a34
I20260811 05:17:06.261260 32342 ts_manager.cc:194] Registered new tserver with Master: 3217a94ca9414c8ea135d593030a6ad2 (127.28.143.129:41539)
I20260811 05:17:06.262058 32342 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.143.129:60659
I20260811 05:17:06.262106 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.130:0
--local_ip_for_outbound_sockets=127.28.143.130
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:38073
--builtin_ntp_servers=127.28.143.148:38121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 05:17:06.347604 32686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:06.347847 32686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:06.347872 32686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:06.349166 32686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:06.349216 32686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:06.349243 32686 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.130
I20260811 05:17:06.350673 32686 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:38121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.143.130
--webserver_port=0
--tserver_master_addrs=127.28.143.190:38073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32686
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.130
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:17:06.350922 32686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:06.351217 32686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:06.356693 32704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:06.356696 32696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:06.356722 32698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:06.356755 32686 server_base.cc:1034] running on GCE node
I20260811 05:17:06.357120 32686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:06.357398 32686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:06.358564 32686 hybrid_clock.cc:648] HybridClock initialized: now 1786425426358548 us; error 29 us; skew 500 ppm
I20260811 05:17:06.359987 32686 webserver.cc:460] Webserver started at http://127.28.143.130:45099/ using document root <none> and password file <none>
I20260811 05:17:06.360251 32686 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:06.360306 32686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:06.360445 32686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:06.361608 32686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-1/data/instance:
uuid: "3f702873578946918047090eb20b5372"
format_stamp: "Formatted at 2026-08-11 05:17:06 on dist-test-slave-f7th"
server_key: "eb87a42f5186110fa61c07822232ee80"
server_key_iv: "54feaec8fcaf895b851ee089bee0a843"
server_key_version: "encryptionkey@0"
I20260811 05:17:06.361951 32686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance:
uuid: "3f702873578946918047090eb20b5372"
format_stamp: "Formatted at 2026-08-11 05:17:06 on dist-test-slave-f7th"
server_key: "eb87a42f5186110fa61c07822232ee80"
server_key_iv: "54feaec8fcaf895b851ee089bee0a843"
server_key_version: "encryptionkey@0"
I20260811 05:17:06.363271 32686 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 05:17:06.364092 32717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:06.364254 32686 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:17:06.364346 32686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-1/wal
uuid: "3f702873578946918047090eb20b5372"
format_stamp: "Formatted at 2026-08-11 05:17:06 on dist-test-slave-f7th"
server_key: "eb87a42f5186110fa61c07822232ee80"
server_key_iv: "54feaec8fcaf895b851ee089bee0a843"
server_key_version: "encryptionkey@0"
I20260811 05:17:06.364410 32686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:06.381687 32686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:06.382040 32686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:06.382179 32686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:06.382407 32686 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:17:06.382754 32686 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:17:06.382789 32686 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:06.382820 32686 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:17:06.382843 32686 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:06.388549 32686 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.130:38219
I20260811 05:17:06.388640 454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.130:38219 every 8 connection(s)
I20260811 05:17:06.389034 32686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-1/data/info.pb
I20260811 05:17:06.394474 32451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:06.389800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.130:48415 (local address 127.28.143.190:38073)
0811 05:17:06.389865 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:06.389868 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:06.390231 (+ 363us) server_negotiation.cc:407] Connection header received
0811 05:17:06.390681 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:06.390683 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:06.390694 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:06.390751 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:06.391419 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:06.391959 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:06.393044 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:06.393238 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:06.393303 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:06.394225 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:06.394226 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:06.394227 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:06.394232 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:06.394248 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:06.394252 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:06.394306 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:06.394376 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 05:17:06.394405 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 05:17:06.394676 460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:06.389788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:38073 (local address 127.28.143.130:48415)
0811 05:17:06.390070 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:06.390107 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:06.390592 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:06.390808 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:06.390812 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:06.390821 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:06.391318 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:06.391323 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:06.392091 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:06.392092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:06.392939 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:06.392942 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:06.393328 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:06.393329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:06.393452 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:06.394020 (+ 568us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:17:06.394020 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:06.394085 (+ 65us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:17:06.394110 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:06.394313 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:06.394316 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:06.394319 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 05:17:06.394366 (+ 47us) client_negotiation.cc:240] Negotiation successful
0811 05:17:06.394455 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":142,"threads_started":1}
I20260811 05:17:06.395345 458 heartbeater.cc:347] Connected to a master server at 127.28.143.190:38073
I20260811 05:17:06.395452 458 heartbeater.cc:464] Registering TS with master...
I20260811 05:17:06.395627 458 heartbeater.cc:511] Master 127.28.143.190:38073 requested a full tablet report, sending...
I20260811 05:17:06.396036 32342 ts_manager.cc:194] Registered new tserver with Master: 3f702873578946918047090eb20b5372 (127.28.143.130:38219)
I20260811 05:17:06.396402 32342 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.143.130:48415
I20260811 05:17:06.399019 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 32686
I20260811 05:17:06.399121 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-1/wal/instance
I20260811 05:17:06.399274 29246 external_mini_cluster.cc:1428] Setting key c1ad8e057bac3b258c362da80818c4aa
I20260811 05:17:06.400444 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.143.131:0
--local_ip_for_outbound_sockets=127.28.143.131
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:38073
--builtin_ntp_servers=127.28.143.148:38121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 05:17:06.486213 461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:06.486457 461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:06.486482 461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:06.487752 461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:06.487805 461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:06.487830 461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.143.131
I20260811 05:17:06.489264 461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:38121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.143.131
--webserver_port=0
--tserver_master_addrs=127.28.143.190:38073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.143.131
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:17:06.489516 461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:06.489814 461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:06.495003 472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:06.495033 461 server_base.cc:1034] running on GCE node
W20260811 05:17:06.495005 473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:06.495062 475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:06.495415 461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:06.495644 461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:06.496783 461 hybrid_clock.cc:648] HybridClock initialized: now 1786425426496771 us; error 33 us; skew 500 ppm
I20260811 05:17:06.498072 461 webserver.cc:460] Webserver started at http://127.28.143.131:41305/ using document root <none> and password file <none>
I20260811 05:17:06.498299 461 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:06.498344 461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:06.498492 461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:06.499611 461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-2/data/instance:
uuid: "f159967da7ec4b8c9de16bd9816ab37c"
format_stamp: "Formatted at 2026-08-11 05:17:06 on dist-test-slave-f7th"
server_key: "2755b8d455d40d29670f335164b7e64a"
server_key_iv: "bc45cd8d0f53782aaa8ea745bf34b5d0"
server_key_version: "encryptionkey@0"
I20260811 05:17:06.499938 461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance:
uuid: "f159967da7ec4b8c9de16bd9816ab37c"
format_stamp: "Formatted at 2026-08-11 05:17:06 on dist-test-slave-f7th"
server_key: "2755b8d455d40d29670f335164b7e64a"
server_key_iv: "bc45cd8d0f53782aaa8ea745bf34b5d0"
server_key_version: "encryptionkey@0"
I20260811 05:17:06.501386 461 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 05:17:06.502226 487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:06.502372 461 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 05:17:06.502463 461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-2/wal
uuid: "f159967da7ec4b8c9de16bd9816ab37c"
format_stamp: "Formatted at 2026-08-11 05:17:06 on dist-test-slave-f7th"
server_key: "2755b8d455d40d29670f335164b7e64a"
server_key_iv: "bc45cd8d0f53782aaa8ea745bf34b5d0"
server_key_version: "encryptionkey@0"
I20260811 05:17:06.502540 461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:06.510900 461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:06.511229 461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:06.511353 461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:06.511595 461 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:17:06.511947 461 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:17:06.511983 461 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:06.512015 461 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:17:06.512041 461 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:06.517747 461 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.131:34075
I20260811 05:17:06.517798 679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.131:34075 every 8 connection(s)
I20260811 05:17:06.518148 461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-2/data/info.pb
I20260811 05:17:06.523659 32451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:06.518921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.131:54053 (local address 127.28.143.190:38073)
0811 05:17:06.518987 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:06.518989 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:06.519222 (+ 233us) server_negotiation.cc:407] Connection header received
0811 05:17:06.519771 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:06.519772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:06.519786 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:06.519839 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:06.520560 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:06.521159 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:06.522214 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:06.522422 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:06.522483 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:06.523370 (+ 887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:06.523372 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:06.523372 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:06.523378 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:06.523394 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:06.523399 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:06.523443 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:06.523530 (+ 87us) server_negotiation.cc:299] Negotiation successful
0811 05:17:06.523566 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 05:17:06.523901 687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:06.518887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:38073 (local address 127.28.143.131:54053)
0811 05:17:06.519097 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:06.519135 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:06.519659 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:06.519914 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:06.519918 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:06.519927 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:06.520456 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:06.520461 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:06.521266 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:06.521267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:06.522109 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:06.522112 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:06.522502 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:06.522504 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:06.522645 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:06.523185 (+ 540us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:17:06.523186 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:06.523217 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:17:06.523245 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:06.523453 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:06.523460 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:06.523464 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:17:06.523526 (+ 62us) client_negotiation.cc:240] Negotiation successful
0811 05:17:06.523659 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":89,"threads_started":1}
I20260811 05:17:06.524410 682 heartbeater.cc:347] Connected to a master server at 127.28.143.190:38073
I20260811 05:17:06.524513 682 heartbeater.cc:464] Registering TS with master...
I20260811 05:17:06.524726 682 heartbeater.cc:511] Master 127.28.143.190:38073 requested a full tablet report, sending...
I20260811 05:17:06.525141 32342 ts_manager.cc:194] Registered new tserver with Master: f159967da7ec4b8c9de16bd9816ab37c (127.28.143.131:34075)
I20260811 05:17:06.525528 32342 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.143.131:54053
I20260811 05:17:06.526886 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 461
I20260811 05:17:06.526980 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-2/wal/instance
I20260811 05:17:06.527191 29246 external_mini_cluster.cc:1428] Setting key 0d7f92fe7ffe27034d25197b4e9dcc60
I20260811 05:17:06.528151 29246 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:17:06.532526 32451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:06.529619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41782 (local address 127.28.143.190:38073)
0811 05:17:06.529715 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:06.529717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:06.529809 (+ 92us) server_negotiation.cc:407] Connection header received
0811 05:17:06.529879 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:06.529880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:06.529894 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:06.529936 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:06.530092 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:06.530584 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:06.531338 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:06.531539 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:06.531593 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:06.532281 (+ 688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:06.532283 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:06.532283 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:06.532288 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:06.532305 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:06.532309 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:06.532355 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:06.532412 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 05:17:06.532440 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 05:17:06.534327 32342 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41782:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:17:06.536026 32342 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:17:06.542992 706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:06.538570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59534 (local address 127.28.143.129:41539)
0811 05:17:06.539260 (+ 690us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:06.539264 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:06.539274 (+ 10us) server_negotiation.cc:407] Connection header received
0811 05:17:06.539309 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:06.539311 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:06.539325 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:06.539559 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:06.539607 (+ 48us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:06.540322 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:06.541783 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:06.542587 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:06.542673 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:06.542779 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:06.542864 (+ 85us) server_negotiation.cc:299] Negotiation successful
0811 05:17:06.542908 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":649,"thread_start_us":104,"threads_started":1}
I20260811 05:17:06.543005 701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:06.538456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:41539 (local address 127.0.0.1:59534)
0811 05:17:06.538645 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:06.538687 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:06.538756 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:06.539431 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:06.539432 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:06.539434 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:06.539486 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:06.539488 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:06.540402 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:06.540403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:06.541703 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:06.541706 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:06.542676 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:06.542676 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:06.542759 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:06.542775 (+ 16us) client_negotiation.cc:769] Sending connection context
0811 05:17:06.542860 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 05:17:06.542908 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":73,"threads_started":1}
I20260811 05:17:06.543241 702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:06.538466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:38219 (local address 127.0.0.1:52210)
0811 05:17:06.538856 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:06.538864 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:06.538897 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:06.539395 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:06.539395 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:06.539397 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:06.539437 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:06.539439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:06.540767 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:06.540768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:06.541817 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:06.541819 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:06.542684 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:06.542686 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:06.542770 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:06.542777 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 05:17:06.542826 (+ 49us) client_negotiation.cc:240] Negotiation successful
0811 05:17:06.542885 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":82,"threads_started":1}
I20260811 05:17:06.543462 711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:06.538982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52210 (local address 127.28.143.130:38219)
0811 05:17:06.539226 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:06.539231 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:06.539241 (+ 10us) server_negotiation.cc:407] Connection header received
0811 05:17:06.539276 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:06.539278 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:06.539293 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:06.539510 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:06.539568 (+ 58us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:06.540698 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:06.541904 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:06.542598 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:06.542969 (+ 371us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:06.543218 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:06.543245 (+ 27us) server_negotiation.cc:299] Negotiation successful
0811 05:17:06.543348 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":110,"threads_started":1}
I20260811 05:17:06.543812 32565 tablet_service.cc:1467] Processing CreateTablet for tablet ec956459d6964413b189477171164b69 (DEFAULT_TABLE table=test-table [id=b1adf318e5ed4e24bf4f7b239da37d80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:17:06.544114 700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:06.538466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.131:34075 (local address 127.0.0.1:48260)
0811 05:17:06.538754 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:06.538763 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:06.538818 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:06.539257 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:06.539261 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:06.539264 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:06.539343 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:06.539345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:06.541891 (+ 2546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:06.541891 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:06.542532 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:06.542533 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:06.543894 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:06.543895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:06.543947 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:06.543950 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 05:17:06.543997 (+ 47us) client_negotiation.cc:240] Negotiation successful
0811 05:17:06.544044 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"spinlock_wait_cycles":768,"thread_start_us":94,"threads_started":1}
I20260811 05:17:06.544149 710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:06.538729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48260 (local address 127.28.143.131:34075)
0811 05:17:06.538910 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:06.538915 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:06.538927 (+ 12us) server_negotiation.cc:407] Connection header received
0811 05:17:06.538969 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:06.538971 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:06.538987 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:06.539080 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:06.539465 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:06.540685 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:06.542614 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:06.543625 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:06.543694 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:06.543781 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:06.544029 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 05:17:06.544072 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":96,"threads_started":1}
I20260811 05:17:06.544256 32565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec956459d6964413b189477171164b69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:06.544260 342 tablet_service.cc:1467] Processing CreateTablet for tablet ec956459d6964413b189477171164b69 (DEFAULT_TABLE table=test-table [id=b1adf318e5ed4e24bf4f7b239da37d80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:17:06.544606 342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec956459d6964413b189477171164b69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:06.544862 571 tablet_service.cc:1467] Processing CreateTablet for tablet ec956459d6964413b189477171164b69 (DEFAULT_TABLE table=test-table [id=b1adf318e5ed4e24bf4f7b239da37d80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:17:06.545202 571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec956459d6964413b189477171164b69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:06.547678 712 tablet_bootstrap.cc:492] T ec956459d6964413b189477171164b69 P f159967da7ec4b8c9de16bd9816ab37c: Bootstrap starting.
I20260811 05:17:06.548180 713 tablet_bootstrap.cc:492] T ec956459d6964413b189477171164b69 P 3217a94ca9414c8ea135d593030a6ad2: Bootstrap starting.
I20260811 05:17:06.548467 712 tablet_bootstrap.cc:654] T ec956459d6964413b189477171164b69 P f159967da7ec4b8c9de16bd9816ab37c: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:06.548784 713 tablet_bootstrap.cc:654] T ec956459d6964413b189477171164b69 P 3217a94ca9414c8ea135d593030a6ad2: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:06.548775 712 log.cc:824] T ec956459d6964413b189477171164b69 P f159967da7ec4b8c9de16bd9816ab37c: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:06.549103 713 log.cc:824] T ec956459d6964413b189477171164b69 P 3217a94ca9414c8ea135d593030a6ad2: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:06.549485 714 tablet_bootstrap.cc:492] T ec956459d6964413b189477171164b69 P 3f702873578946918047090eb20b5372: Bootstrap starting.
I20260811 05:17:06.549584 712 tablet_bootstrap.cc:492] T ec956459d6964413b189477171164b69 P f159967da7ec4b8c9de16bd9816ab37c: No bootstrap required, opened a new log
I20260811 05:17:06.549660 712 ts_tablet_manager.cc:1397] T ec956459d6964413b189477171164b69 P f159967da7ec4b8c9de16bd9816ab37c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 05:17:06.550055 714 tablet_bootstrap.cc:654] T ec956459d6964413b189477171164b69 P 3f702873578946918047090eb20b5372: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:06.550324 714 log.cc:824] T ec956459d6964413b189477171164b69 P 3f702873578946918047090eb20b5372: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:06.550846 713 tablet_bootstrap.cc:492] T ec956459d6964413b189477171164b69 P 3217a94ca9414c8ea135d593030a6ad2: No bootstrap required, opened a new log
I20260811 05:17:06.550922 713 ts_tablet_manager.cc:1397] T ec956459d6964413b189477171164b69 P 3217a94ca9414c8ea135d593030a6ad2: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260811 05:17:06.551759 712 raft_consensus.cc:348] T ec956459d6964413b189477171164b69 P f159967da7ec4b8c9de16bd9816ab37c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f159967da7ec4b8c9de16bd9816ab37c" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 34075 } } peers { permanent_uuid: "3f702873578946918047090eb20b5372" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 38219 } } peers { permanent_uuid: "3217a94ca9414c8ea135d593030a6ad2" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 41539 } }
I20260811 05:17:06.551937 712 raft_consensus.cc:374] T ec956459d6964413b189477171164b69 P f159967da7ec4b8c9de16bd9816ab37c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:06.551981 712 raft_consensus.cc:729] T ec956459d6964413b189477171164b69 P f159967da7ec4b8c9de16bd9816ab37c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f159967da7ec4b8c9de16bd9816ab37c, State: Initialized, Role: FOLLOWER
I20260811 05:17:06.552106 712 consensus_queue.cc:260] T ec956459d6964413b189477171164b69 P f159967da7ec4b8c9de16bd9816ab37c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f159967da7ec4b8c9de16bd9816ab37c" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 34075 } } peers { permanent_uuid: "3f702873578946918047090eb20b5372" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 38219 } } peers { permanent_uuid: "3217a94ca9414c8ea135d593030a6ad2" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 41539 } }
I20260811 05:17:06.552364 712 ts_tablet_manager.cc:1428] T ec956459d6964413b189477171164b69 P f159967da7ec4b8c9de16bd9816ab37c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 05:17:06.552439 682 heartbeater.cc:503] Master 127.28.143.190:38073 was elected leader, sending a full tablet report...
I20260811 05:17:06.553241 713 raft_consensus.cc:348] T ec956459d6964413b189477171164b69 P 3217a94ca9414c8ea135d593030a6ad2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f159967da7ec4b8c9de16bd9816ab37c" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 34075 } } peers { permanent_uuid: "3f702873578946918047090eb20b5372" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 38219 } } peers { permanent_uuid: "3217a94ca9414c8ea135d593030a6ad2" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 41539 } }
I20260811 05:17:06.553390 713 raft_consensus.cc:374] T ec956459d6964413b189477171164b69 P 3217a94ca9414c8ea135d593030a6ad2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:06.553423 713 raft_consensus.cc:729] T ec956459d6964413b189477171164b69 P 3217a94ca9414c8ea135d593030a6ad2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3217a94ca9414c8ea135d593030a6ad2, State: Initialized, Role: FOLLOWER
I20260811 05:17:06.553561 713 consensus_queue.cc:260] T ec956459d6964413b189477171164b69 P 3217a94ca9414c8ea135d593030a6ad2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f159967da7ec4b8c9de16bd9816ab37c" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 34075 } } peers { permanent_uuid: "3f702873578946918047090eb20b5372" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 38219 } } peers { permanent_uuid: "3217a94ca9414c8ea135d593030a6ad2" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 41539 } }
I20260811 05:17:06.553851 713 ts_tablet_manager.cc:1428] T ec956459d6964413b189477171164b69 P 3217a94ca9414c8ea135d593030a6ad2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 05:17:06.553903 32681 heartbeater.cc:503] Master 127.28.143.190:38073 was elected leader, sending a full tablet report...
I20260811 05:17:06.554385 714 tablet_bootstrap.cc:492] T ec956459d6964413b189477171164b69 P 3f702873578946918047090eb20b5372: No bootstrap required, opened a new log
I20260811 05:17:06.554458 714 ts_tablet_manager.cc:1397] T ec956459d6964413b189477171164b69 P 3f702873578946918047090eb20b5372: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260811 05:17:06.556360 714 raft_consensus.cc:348] T ec956459d6964413b189477171164b69 P 3f702873578946918047090eb20b5372 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f159967da7ec4b8c9de16bd9816ab37c" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 34075 } } peers { permanent_uuid: "3f702873578946918047090eb20b5372" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 38219 } } peers { permanent_uuid: "3217a94ca9414c8ea135d593030a6ad2" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 41539 } }
I20260811 05:17:06.556514 714 raft_consensus.cc:374] T ec956459d6964413b189477171164b69 P 3f702873578946918047090eb20b5372 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:06.556547 714 raft_consensus.cc:729] T ec956459d6964413b189477171164b69 P 3f702873578946918047090eb20b5372 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f702873578946918047090eb20b5372, State: Initialized, Role: FOLLOWER
I20260811 05:17:06.556702 714 consensus_queue.cc:260] T ec956459d6964413b189477171164b69 P 3f702873578946918047090eb20b5372 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f159967da7ec4b8c9de16bd9816ab37c" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 34075 } } peers { permanent_uuid: "3f702873578946918047090eb20b5372" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 38219 } } peers { permanent_uuid: "3217a94ca9414c8ea135d593030a6ad2" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 41539 } }
I20260811 05:17:06.556991 714 ts_tablet_manager.cc:1428] T ec956459d6964413b189477171164b69 P 3f702873578946918047090eb20b5372: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 05:17:06.557055 458 heartbeater.cc:503] Master 127.28.143.190:38073 was elected leader, sending a full tablet report...
W20260811 05:17:06.639608 459 tablet.cc:2369] T ec956459d6964413b189477171164b69 P 3f702873578946918047090eb20b5372: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 05:17:06.754529 32682 tablet.cc:2369] T ec956459d6964413b189477171164b69 P 3217a94ca9414c8ea135d593030a6ad2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 05:17:06.768847 686 tablet.cc:2369] T ec956459d6964413b189477171164b69 P f159967da7ec4b8c9de16bd9816ab37c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:17:07.000478 722 raft_consensus.cc:482] T ec956459d6964413b189477171164b69 P 3217a94ca9414c8ea135d593030a6ad2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:17:07.000669 722 raft_consensus.cc:504] T ec956459d6964413b189477171164b69 P 3217a94ca9414c8ea135d593030a6ad2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f159967da7ec4b8c9de16bd9816ab37c" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 34075 } } peers { permanent_uuid: "3f702873578946918047090eb20b5372" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 38219 } } peers { permanent_uuid: "3217a94ca9414c8ea135d593030a6ad2" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 41539 } }
I20260811 05:17:07.001156 722 leader_election.cc:288] T ec956459d6964413b189477171164b69 P 3217a94ca9414c8ea135d593030a6ad2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f159967da7ec4b8c9de16bd9816ab37c (127.28.143.131:34075), 3f702873578946918047090eb20b5372 (127.28.143.130:38219)
I20260811 05:17:07.004227 711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:07.001464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:48079 (local address 127.28.143.130:38219)
0811 05:17:07.001513 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:07.001515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:07.001614 (+ 99us) server_negotiation.cc:407] Connection header received
0811 05:17:07.001660 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:07.001661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:07.001672 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:07.001725 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:07.001908 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:07.002464 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:07.003559 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:07.003901 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:07.003962 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:07.003972 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:07.004121 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 05:17:07.004159 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 05:17:07.004228 710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:07.001502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.143.129:46981 (local address 127.28.143.131:34075)
0811 05:17:07.001570 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:07.001572 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:07.001638 (+ 66us) server_negotiation.cc:407] Connection header received
0811 05:17:07.001700 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:07.001700 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:07.001709 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:07.001767 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:07.002044 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:07.002546 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:07.003552 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:07.003906 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:07.003968 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:07.003976 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:07.004131 (+ 155us) server_negotiation.cc:299] Negotiation successful
0811 05:17:07.004167 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 05:17:07.004241 728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:07.001347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.130:38219 (local address 127.28.143.129:48079)
0811 05:17:07.001523 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:07.001534 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:07.001606 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:07.001741 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:07.001742 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:07.001744 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:07.001827 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:07.001829 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:07.002570 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:07.002571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:07.003457 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:07.003460 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:07.003983 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:07.003984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:07.004050 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:07.004054 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:17:07.004096 (+ 42us) client_negotiation.cc:240] Negotiation successful
0811 05:17:07.004139 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":34,"thread_start_us":102,"threads_started":1}
I20260811 05:17:07.004254 724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:07.001410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.131:34075 (local address 127.28.143.129:46981)
0811 05:17:07.001558 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:07.001568 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:07.001636 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:07.001841 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:07.001842 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:07.001845 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:07.001937 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:07.001939 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:07.002631 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:07.002632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:07.003456 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:07.003458 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:07.003988 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:07.003989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:07.004054 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:07.004058 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:17:07.004113 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 05:17:07.004154 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":25,"thread_start_us":71,"threads_started":1}
I20260811 05:17:07.004638 376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec956459d6964413b189477171164b69" candidate_uuid: "3217a94ca9414c8ea135d593030a6ad2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f702873578946918047090eb20b5372" is_pre_election: true
I20260811 05:17:07.004657 603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec956459d6964413b189477171164b69" candidate_uuid: "3217a94ca9414c8ea135d593030a6ad2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f159967da7ec4b8c9de16bd9816ab37c" is_pre_election: true
I20260811 05:17:07.004807 376 raft_consensus.cc:2454] T ec956459d6964413b189477171164b69 P 3f702873578946918047090eb20b5372 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3217a94ca9414c8ea135d593030a6ad2 in term 0.
I20260811 05:17:07.004815 603 raft_consensus.cc:2454] T ec956459d6964413b189477171164b69 P f159967da7ec4b8c9de16bd9816ab37c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3217a94ca9414c8ea135d593030a6ad2 in term 0.
I20260811 05:17:07.005064 32482 leader_election.cc:302] T ec956459d6964413b189477171164b69 P 3217a94ca9414c8ea135d593030a6ad2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3217a94ca9414c8ea135d593030a6ad2, 3f702873578946918047090eb20b5372; no voters:
I20260811 05:17:07.005249 722 raft_consensus.cc:2781] T ec956459d6964413b189477171164b69 P 3217a94ca9414c8ea135d593030a6ad2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:17:07.005306 722 raft_consensus.cc:482] T ec956459d6964413b189477171164b69 P 3217a94ca9414c8ea135d593030a6ad2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:17:07.005321 722 raft_consensus.cc:3037] T ec956459d6964413b189477171164b69 P 3217a94ca9414c8ea135d593030a6ad2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:07.006246 722 raft_consensus.cc:504] T ec956459d6964413b189477171164b69 P 3217a94ca9414c8ea135d593030a6ad2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f159967da7ec4b8c9de16bd9816ab37c" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 34075 } } peers { permanent_uuid: "3f702873578946918047090eb20b5372" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 38219 } } peers { permanent_uuid: "3217a94ca9414c8ea135d593030a6ad2" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 41539 } }
I20260811 05:17:07.006386 722 leader_election.cc:288] T ec956459d6964413b189477171164b69 P 3217a94ca9414c8ea135d593030a6ad2 [CANDIDATE]: Term 1 election: Requested vote from peers f159967da7ec4b8c9de16bd9816ab37c (127.28.143.131:34075), 3f702873578946918047090eb20b5372 (127.28.143.130:38219)
I20260811 05:17:07.006592 376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec956459d6964413b189477171164b69" candidate_uuid: "3217a94ca9414c8ea135d593030a6ad2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f702873578946918047090eb20b5372"
I20260811 05:17:07.006632 603 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec956459d6964413b189477171164b69" candidate_uuid: "3217a94ca9414c8ea135d593030a6ad2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f159967da7ec4b8c9de16bd9816ab37c"
I20260811 05:17:07.006690 376 raft_consensus.cc:3037] T ec956459d6964413b189477171164b69 P 3f702873578946918047090eb20b5372 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:07.006701 603 raft_consensus.cc:3037] T ec956459d6964413b189477171164b69 P f159967da7ec4b8c9de16bd9816ab37c [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:07.007611 376 raft_consensus.cc:2454] T ec956459d6964413b189477171164b69 P 3f702873578946918047090eb20b5372 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3217a94ca9414c8ea135d593030a6ad2 in term 1.
I20260811 05:17:07.007611 603 raft_consensus.cc:2454] T ec956459d6964413b189477171164b69 P f159967da7ec4b8c9de16bd9816ab37c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3217a94ca9414c8ea135d593030a6ad2 in term 1.
I20260811 05:17:07.007856 32481 leader_election.cc:302] T ec956459d6964413b189477171164b69 P 3217a94ca9414c8ea135d593030a6ad2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3217a94ca9414c8ea135d593030a6ad2, f159967da7ec4b8c9de16bd9816ab37c; no voters:
I20260811 05:17:07.007961 722 raft_consensus.cc:2781] T ec956459d6964413b189477171164b69 P 3217a94ca9414c8ea135d593030a6ad2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:07.008152 722 raft_consensus.cc:686] T ec956459d6964413b189477171164b69 P 3217a94ca9414c8ea135d593030a6ad2 [term 1 LEADER]: Becoming Leader. State: Replica: 3217a94ca9414c8ea135d593030a6ad2, State: Running, Role: LEADER
I20260811 05:17:07.008263 722 consensus_queue.cc:237] T ec956459d6964413b189477171164b69 P 3217a94ca9414c8ea135d593030a6ad2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f159967da7ec4b8c9de16bd9816ab37c" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 34075 } } peers { permanent_uuid: "3f702873578946918047090eb20b5372" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 38219 } } peers { permanent_uuid: "3217a94ca9414c8ea135d593030a6ad2" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 41539 } }
I20260811 05:17:07.009191 32342 catalog_manager.cc:5495] T ec956459d6964413b189477171164b69 P 3217a94ca9414c8ea135d593030a6ad2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3217a94ca9414c8ea135d593030a6ad2 (127.28.143.129). New cstate: current_term: 1 leader_uuid: "3217a94ca9414c8ea135d593030a6ad2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f159967da7ec4b8c9de16bd9816ab37c" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 34075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f702873578946918047090eb20b5372" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 38219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3217a94ca9414c8ea135d593030a6ad2" member_type: VOTER last_known_addr { host: "127.28.143.129" port: 41539 } health_report { overall_health: HEALTHY } } }
I20260811 05:17:07.099380 740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:07.096871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59544 (local address 127.28.143.129:41539)
0811 05:17:07.096986 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:07.096988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:07.096998 (+ 10us) server_negotiation.cc:407] Connection header received
0811 05:17:07.097061 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:07.097061 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:07.097081 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:07.097151 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:17:07.097255 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:07.097760 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:07.098579 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:07.098788 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:07.098883 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:07.099065 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:17:07.099174 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:17:07.099235 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:07.099264 (+ 29us) server_negotiation.cc:299] Negotiation successful
0811 05:17:07.099284 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":60,"threads_started":1}
I20260811 05:17:07.101164 376 raft_consensus.cc:1261] T ec956459d6964413b189477171164b69 P 3f702873578946918047090eb20b5372 [term 1 FOLLOWER]: Refusing update from remote peer 3217a94ca9414c8ea135d593030a6ad2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:17:07.101178 603 raft_consensus.cc:1261] T ec956459d6964413b189477171164b69 P f159967da7ec4b8c9de16bd9816ab37c [term 1 FOLLOWER]: Refusing update from remote peer 3217a94ca9414c8ea135d593030a6ad2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:17:07.101548 731 consensus_queue.cc:1035] T ec956459d6964413b189477171164b69 P 3217a94ca9414c8ea135d593030a6ad2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f159967da7ec4b8c9de16bd9816ab37c" member_type: VOTER last_known_addr { host: "127.28.143.131" port: 34075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:17:07.101687 722 consensus_queue.cc:1035] T ec956459d6964413b189477171164b69 P 3217a94ca9414c8ea135d593030a6ad2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f702873578946918047090eb20b5372" member_type: VOTER last_known_addr { host: "127.28.143.130" port: 38219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:17:07.104857 756 mvcc.cc:205] Tried to move back new op lower bound from 7317198549403541504 to 7317198549026471936. Current Snapshot: MvccSnapshot[applied={T|T < 7317198549403541504}]
I20260811 05:17:07.105435 755 mvcc.cc:205] Tried to move back new op lower bound from 7317198549403541504 to 7317198549026471936. Current Snapshot: MvccSnapshot[applied={T|T < 7317198549403541504}]
I20260811 05:17:07.107002 32339 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:17:07.107112 32339 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:17:07.108006 32339 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bfc45aed75d3411ba310e80e532a2ffc: Sending DeleteTablet for 3 replicas of tablet ec956459d6964413b189477171164b69
I20260811 05:17:07.108392 342 tablet_service.cc:1514] Processing DeleteTablet for tablet ec956459d6964413b189477171164b69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:17:07 UTC) from {username='slave'} at 127.0.0.1:52210
I20260811 05:17:07.108461 32565 tablet_service.cc:1514] Processing DeleteTablet for tablet ec956459d6964413b189477171164b69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:17:07 UTC) from {username='slave'} at 127.0.0.1:59534
I20260811 05:17:07.108693 571 tablet_service.cc:1514] Processing DeleteTablet for tablet ec956459d6964413b189477171164b69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:17:07 UTC) from {username='slave'} at 127.0.0.1:48260
I20260811 05:17:07.108711 769 tablet_replica.cc:330] T ec956459d6964413b189477171164b69 P 3f702873578946918047090eb20b5372: stopping tablet replica
I20260811 05:17:07.108834 769 raft_consensus.cc:2229] T ec956459d6964413b189477171164b69 P 3f702873578946918047090eb20b5372 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:17:07.108922 774 tablet_replica.cc:330] T ec956459d6964413b189477171164b69 P f159967da7ec4b8c9de16bd9816ab37c: stopping tablet replica
I20260811 05:17:07.108959 769 raft_consensus.cc:2258] T ec956459d6964413b189477171164b69 P 3f702873578946918047090eb20b5372 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:17:07.109063 774 raft_consensus.cc:2229] T ec956459d6964413b189477171164b69 P f159967da7ec4b8c9de16bd9816ab37c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:17:07.109191 774 raft_consensus.cc:2258] T ec956459d6964413b189477171164b69 P f159967da7ec4b8c9de16bd9816ab37c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:17:07.109313 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 32453
I20260811 05:17:07.109426 769 ts_tablet_manager.cc:1905] T ec956459d6964413b189477171164b69 P 3f702873578946918047090eb20b5372: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:17:07.110076 774 ts_tablet_manager.cc:1905] T ec956459d6964413b189477171164b69 P f159967da7ec4b8c9de16bd9816ab37c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:17:07.111539 769 ts_tablet_manager.cc:1918] T ec956459d6964413b189477171164b69 P 3f702873578946918047090eb20b5372: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:17:07.111642 769 log.cc:1196] T ec956459d6964413b189477171164b69 P 3f702873578946918047090eb20b5372: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-1/wal/wals/ec956459d6964413b189477171164b69
I20260811 05:17:07.111589 774 ts_tablet_manager.cc:1918] T ec956459d6964413b189477171164b69 P f159967da7ec4b8c9de16bd9816ab37c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:17:07.111819 774 log.cc:1196] T ec956459d6964413b189477171164b69 P f159967da7ec4b8c9de16bd9816ab37c: Deleting WAL directory at /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425159060450-29246-0/minicluster-data/ts-2/wal/wals/ec956459d6964413b189477171164b69
I20260811 05:17:07.111992 769 ts_tablet_manager.cc:1939] T ec956459d6964413b189477171164b69 P 3f702873578946918047090eb20b5372: Deleting consensus metadata
I20260811 05:17:07.112213 774 ts_tablet_manager.cc:1939] T ec956459d6964413b189477171164b69 P f159967da7ec4b8c9de16bd9816ab37c: Deleting consensus metadata
I20260811 05:17:07.112598 32317 catalog_manager.cc:4841] TS 3f702873578946918047090eb20b5372 (127.28.143.130:38219): tablet ec956459d6964413b189477171164b69 (table test-table [id=b1adf318e5ed4e24bf4f7b239da37d80]) successfully deleted
I20260811 05:17:07.112937 32315 catalog_manager.cc:4841] TS f159967da7ec4b8c9de16bd9816ab37c (127.28.143.131:34075): tablet ec956459d6964413b189477171164b69 (table test-table [id=b1adf318e5ed4e24bf4f7b239da37d80]) successfully deleted
W20260811 05:17:07.120698 32311 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.143.129:41539 (error 108)
I20260811 05:17:07.121371 776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:07.121198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.129:41539 (local address 127.0.0.1:59548)
0811 05:17:07.121285 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:07.121330 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.129:41539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67,"thread_start_us":46,"threads_started":1}
I20260811 05:17:07.121353 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 32686
W20260811 05:17:07.124650 32311 catalog_manager.cc:4584] TS 3217a94ca9414c8ea135d593030a6ad2 (127.28.143.129:41539): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ec956459d6964413b189477171164b69: Network error: Client connection negotiation failed: client connection to 127.28.143.129:41539: connect: Connection refused (error 111)
I20260811 05:17:07.141342 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 461
I20260811 05:17:07.159792 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 32285
2026-08-11T05:17:07Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1263 ms)
[----------] 6 tests from EncryptionPolicyTest (10075 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425159060450-29246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:17:07 dist-test-slave-f7th krb5kdc[780](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:17:07 dist-test-slave-f7th krb5kdc[780](info): set up 2 sockets
Aug 11 05:17:07 dist-test-slave-f7th krb5kdc[780](info): commencing operation
krb5kdc: starting...
W20260811 05:17:08.381242 29246 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Aug 11 05:17:08 dist-test-slave-f7th krb5kdc[780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:17:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:17:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 05:17:08.516350 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46331
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46331
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:08.602452 800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:08.602690 800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:08.602715 800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:08.603941 800 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:08.603994 800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:08.604007 800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:08.604018 800 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:08.604030 800 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:08.605676 800 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46331
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46331
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:17:08.605970 800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:08.606210 800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:08.611315 812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:08.611317 811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:08.611316 816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:08.612020 800 server_base.cc:1034] running on GCE node
I20260811 05:17:08.612301 800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:08.612638 800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:08.613816 800 hybrid_clock.cc:648] HybridClock initialized: now 1786425428613795 us; error 37 us; skew 500 ppm
Aug 11 05:17:08 dist-test-slave-f7th krb5kdc[780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425428, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:08.616354 800 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 05:17:08.616894 800 webserver.cc:460] Webserver started at http://127.0.0.1:43207/ using document root <none> and password file <none>
I20260811 05:17:08.617115 800 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:08.617166 800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:08.617297 800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:08.618381 800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "25ee0f293e5845ec87af8ea85df86ba0"
format_stamp: "Formatted at 2026-08-11 05:17:08 on dist-test-slave-f7th"
server_key: "a7133d6ae6565bcac5d40cab689f06d9"
server_key_iv: "418d713d086470192bed3807e24d8eb8"
server_key_version: "encryptionkey@0"
I20260811 05:17:08.618777 800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "25ee0f293e5845ec87af8ea85df86ba0"
format_stamp: "Formatted at 2026-08-11 05:17:08 on dist-test-slave-f7th"
server_key: "a7133d6ae6565bcac5d40cab689f06d9"
server_key_iv: "418d713d086470192bed3807e24d8eb8"
server_key_version: "encryptionkey@0"
I20260811 05:17:08.620152 800 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 05:17:08.620970 827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:08.621112 800 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 05:17:08.621206 800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "25ee0f293e5845ec87af8ea85df86ba0"
format_stamp: "Formatted at 2026-08-11 05:17:08 on dist-test-slave-f7th"
server_key: "a7133d6ae6565bcac5d40cab689f06d9"
server_key_iv: "418d713d086470192bed3807e24d8eb8"
server_key_version: "encryptionkey@0"
I20260811 05:17:08.621270 800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:08.636524 800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:08.639026 800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:08.639207 800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:08.643544 800 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46331
I20260811 05:17:08.643587 936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46331 every 8 connection(s)
I20260811 05:17:08.643954 800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:17:08.644976 939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:08.647511 939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25ee0f293e5845ec87af8ea85df86ba0: Bootstrap starting.
I20260811 05:17:08.648129 939 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25ee0f293e5845ec87af8ea85df86ba0: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:08.648388 939 log.cc:824] T 00000000000000000000000000000000 P 25ee0f293e5845ec87af8ea85df86ba0: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:08.649184 939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25ee0f293e5845ec87af8ea85df86ba0: No bootstrap required, opened a new log
I20260811 05:17:08.650813 939 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25ee0f293e5845ec87af8ea85df86ba0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25ee0f293e5845ec87af8ea85df86ba0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46331 } }
I20260811 05:17:08.650966 939 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25ee0f293e5845ec87af8ea85df86ba0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:08.650990 939 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25ee0f293e5845ec87af8ea85df86ba0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25ee0f293e5845ec87af8ea85df86ba0, State: Initialized, Role: FOLLOWER
I20260811 05:17:08.651109 939 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25ee0f293e5845ec87af8ea85df86ba0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25ee0f293e5845ec87af8ea85df86ba0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46331 } }
I20260811 05:17:08.651163 939 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25ee0f293e5845ec87af8ea85df86ba0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:17:08.651189 939 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25ee0f293e5845ec87af8ea85df86ba0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:17:08.651221 939 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25ee0f293e5845ec87af8ea85df86ba0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:08.651762 939 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25ee0f293e5845ec87af8ea85df86ba0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25ee0f293e5845ec87af8ea85df86ba0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46331 } }
I20260811 05:17:08.651856 939 leader_election.cc:302] T 00000000000000000000000000000000 P 25ee0f293e5845ec87af8ea85df86ba0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25ee0f293e5845ec87af8ea85df86ba0; no voters:
I20260811 05:17:08.652004 939 leader_election.cc:288] T 00000000000000000000000000000000 P 25ee0f293e5845ec87af8ea85df86ba0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:17:08.652076 942 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25ee0f293e5845ec87af8ea85df86ba0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:08.652273 939 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25ee0f293e5845ec87af8ea85df86ba0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:08.652287 942 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25ee0f293e5845ec87af8ea85df86ba0 [term 1 LEADER]: Becoming Leader. State: Replica: 25ee0f293e5845ec87af8ea85df86ba0, State: Running, Role: LEADER
I20260811 05:17:08.652405 942 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25ee0f293e5845ec87af8ea85df86ba0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25ee0f293e5845ec87af8ea85df86ba0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46331 } }
I20260811 05:17:08.652922 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 800
I20260811 05:17:08.653060 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:17:08.653013 944 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25ee0f293e5845ec87af8ea85df86ba0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25ee0f293e5845ec87af8ea85df86ba0. Latest consensus state: current_term: 1 leader_uuid: "25ee0f293e5845ec87af8ea85df86ba0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25ee0f293e5845ec87af8ea85df86ba0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46331 } } }
I20260811 05:17:08.653090 944 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25ee0f293e5845ec87af8ea85df86ba0 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:08.653275 29246 external_mini_cluster.cc:1428] Setting key 8d391740cc7c71e0effe268142b52cf3
I20260811 05:17:08.653419 943 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25ee0f293e5845ec87af8ea85df86ba0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "25ee0f293e5845ec87af8ea85df86ba0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25ee0f293e5845ec87af8ea85df86ba0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46331 } } }
I20260811 05:17:08.653487 943 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25ee0f293e5845ec87af8ea85df86ba0 [sys.catalog]: This master's current role is: LEADER
W20260811 05:17:08.654400 965 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 25ee0f293e5845ec87af8ea85df86ba0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:17:08.654510 965 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:17:08.654628 955 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:17:08.654809 955 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:17:08.656378 955 catalog_manager.cc:1347] Generated new cluster ID: 8719322b367041d4992e3a7f29798dfa
I20260811 05:17:08.656431 955 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 05:17:08 dist-test-slave-f7th krb5kdc[780](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 05:17:08.665593 976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:08.653795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43349 (local address 127.0.0.1:46331)
0811 05:17:08.654107 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:08.654117 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:08.654152 (+ 35us) server_negotiation.cc:407] Connection header received
0811 05:17:08.654286 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:08.654294 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:08.654357 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:08.654510 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:08.655311 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:08.656566 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:08.657756 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:08.658378 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:08.658478 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:08.661477 (+ 2999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:08.661482 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:08.661483 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:08.661500 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:08.663523 (+ 2023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:08.664025 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:08.664029 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:08.664030 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:08.664147 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:08.664372 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:08.664372 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:08.664373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:08.664776 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:08.664983 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:08.665171 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 05:17:08.665295 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":65,"threads_started":1}
I20260811 05:17:08.666684 29246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:17:08.667768 955 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:17:08.668457 955 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:17:08.676990 976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:08.668268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34447 (local address 127.0.0.1:46331)
0811 05:17:08.668315 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:08.668317 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:08.668526 (+ 209us) server_negotiation.cc:407] Connection header received
0811 05:17:08.668579 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:08.668580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:08.668591 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:08.668679 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:08.669388 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:08.670236 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:08.671273 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:08.671630 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:08.671705 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:08.673837 (+ 2132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:08.673839 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:08.673839 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:08.673849 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:08.675640 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:08.676126 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:08.676127 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:08.676128 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:08.676197 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:08.676397 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:08.676397 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:08.676398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:08.676545 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:08.676715 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:08.676810 (+ 95us) server_negotiation.cc:299] Negotiation successful
0811 05:17:08.676882 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 05:17:08.678071 955 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25ee0f293e5845ec87af8ea85df86ba0: Generated new TSK 0
I20260811 05:17:08.678254 955 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:17:08.681872 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 800
W20260811 05:17:08.712951 777 connection.cc:535] client connection to 127.0.0.1:46331 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46331): Connection reset by peer (error 104)
2026-08-11T05:17:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1541 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425159060450-29246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:17:08 dist-test-slave-f7th krb5kdc[994](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:17:08 dist-test-slave-f7th krb5kdc[994](info): set up 2 sockets
Aug 11 05:17:08 dist-test-slave-f7th krb5kdc[994](info): commencing operation
krb5kdc: starting...
W20260811 05:17:09.908540 29246 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Aug 11 05:17:10 dist-test-slave-f7th krb5kdc[994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425430, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:17:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:17:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 05:17:10.045341 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38777
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38777
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:10.131631 1018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:10.131867 1018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:10.131903 1018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:10.133234 1018 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:10.133289 1018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:10.133303 1018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:10.133315 1018 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:10.133337 1018 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:10.135044 1018 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38777
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1018
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:17:10.135319 1018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:10.135556 1018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:10.140717 1025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:10.140771 1029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:10.140734 1026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:10.141158 1018 server_base.cc:1034] running on GCE node
I20260811 05:17:10.141465 1018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:10.141758 1018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:10.142943 1018 hybrid_clock.cc:648] HybridClock initialized: now 1786425430142928 us; error 32 us; skew 500 ppm
Aug 11 05:17:10 dist-test-slave-f7th krb5kdc[994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425430, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:10.145701 1018 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 05:17:10.146178 1018 webserver.cc:460] Webserver started at http://127.0.0.1:36299/ using document root <none> and password file <none>
I20260811 05:17:10.146442 1018 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:10.146497 1018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:10.146633 1018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:10.147758 1018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "cbe400f6aec3476cbb74961287fbc08e"
format_stamp: "Formatted at 2026-08-11 05:17:10 on dist-test-slave-f7th"
server_key: "9009679c3c94d2e01bcb67e14f8d9005"
server_key_iv: "e31eada309e7f1e5b2458c18720a2476"
server_key_version: "encryptionkey@0"
I20260811 05:17:10.148115 1018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "cbe400f6aec3476cbb74961287fbc08e"
format_stamp: "Formatted at 2026-08-11 05:17:10 on dist-test-slave-f7th"
server_key: "9009679c3c94d2e01bcb67e14f8d9005"
server_key_iv: "e31eada309e7f1e5b2458c18720a2476"
server_key_version: "encryptionkey@0"
I20260811 05:17:10.149616 1018 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 05:17:10.150413 1043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:10.150604 1018 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:17:10.150702 1018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "cbe400f6aec3476cbb74961287fbc08e"
format_stamp: "Formatted at 2026-08-11 05:17:10 on dist-test-slave-f7th"
server_key: "9009679c3c94d2e01bcb67e14f8d9005"
server_key_iv: "e31eada309e7f1e5b2458c18720a2476"
server_key_version: "encryptionkey@0"
I20260811 05:17:10.150779 1018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:10.160946 1018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:10.163244 1018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:10.163404 1018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:10.167462 1018 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38777
I20260811 05:17:10.167513 1132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38777 every 8 connection(s)
I20260811 05:17:10.167871 1018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:17:10.168931 1133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:10.171429 1133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbe400f6aec3476cbb74961287fbc08e: Bootstrap starting.
I20260811 05:17:10.171865 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 1018
I20260811 05:17:10.172024 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:17:10.172212 29246 external_mini_cluster.cc:1428] Setting key ba234db616bef8ca31e14dcb65a7ba2f
I20260811 05:17:10.172354 1133 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cbe400f6aec3476cbb74961287fbc08e: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:10.172698 1133 log.cc:824] T 00000000000000000000000000000000 P cbe400f6aec3476cbb74961287fbc08e: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:10.173588 1133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbe400f6aec3476cbb74961287fbc08e: No bootstrap required, opened a new log
I20260811 05:17:10.176081 1133 raft_consensus.cc:348] T 00000000000000000000000000000000 P cbe400f6aec3476cbb74961287fbc08e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe400f6aec3476cbb74961287fbc08e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38777 } }
I20260811 05:17:10.176235 1133 raft_consensus.cc:374] T 00000000000000000000000000000000 P cbe400f6aec3476cbb74961287fbc08e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:10.176257 1133 raft_consensus.cc:729] T 00000000000000000000000000000000 P cbe400f6aec3476cbb74961287fbc08e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbe400f6aec3476cbb74961287fbc08e, State: Initialized, Role: FOLLOWER
I20260811 05:17:10.176373 1133 consensus_queue.cc:260] T 00000000000000000000000000000000 P cbe400f6aec3476cbb74961287fbc08e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe400f6aec3476cbb74961287fbc08e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38777 } }
I20260811 05:17:10.176453 1133 raft_consensus.cc:388] T 00000000000000000000000000000000 P cbe400f6aec3476cbb74961287fbc08e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:17:10.176477 1133 raft_consensus.cc:482] T 00000000000000000000000000000000 P cbe400f6aec3476cbb74961287fbc08e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:17:10.176505 1133 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cbe400f6aec3476cbb74961287fbc08e [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:10.177162 1133 raft_consensus.cc:504] T 00000000000000000000000000000000 P cbe400f6aec3476cbb74961287fbc08e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe400f6aec3476cbb74961287fbc08e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38777 } }
I20260811 05:17:10.177271 1133 leader_election.cc:302] T 00000000000000000000000000000000 P cbe400f6aec3476cbb74961287fbc08e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cbe400f6aec3476cbb74961287fbc08e; no voters:
I20260811 05:17:10.177479 1133 leader_election.cc:288] T 00000000000000000000000000000000 P cbe400f6aec3476cbb74961287fbc08e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:17:10.177538 1141 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cbe400f6aec3476cbb74961287fbc08e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:10.177726 1141 raft_consensus.cc:686] T 00000000000000000000000000000000 P cbe400f6aec3476cbb74961287fbc08e [term 1 LEADER]: Becoming Leader. State: Replica: cbe400f6aec3476cbb74961287fbc08e, State: Running, Role: LEADER
I20260811 05:17:10.177772 1133 sys_catalog.cc:564] T 00000000000000000000000000000000 P cbe400f6aec3476cbb74961287fbc08e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:10.178586 1139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:10.172821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46985 (local address 127.0.0.1:38777)
0811 05:17:10.173076 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:10.173083 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:10.173110 (+ 27us) server_negotiation.cc:407] Connection header received
0811 05:17:10.173696 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:10.173706 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:10.173776 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:10.173934 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:10.174142 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:10.175532 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:10.176407 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:10.177006 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:10.177113 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:10.177913 (+ 800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:10.177918 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:10.177918 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:10.178109 (+ 191us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:10.178143 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:10.178155 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:10.178224 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:10.178362 (+ 138us) server_negotiation.cc:299] Negotiation successful
0811 05:17:10.178395 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":83,"threads_started":1}
I20260811 05:17:10.179214 1141 consensus_queue.cc:237] T 00000000000000000000000000000000 P cbe400f6aec3476cbb74961287fbc08e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe400f6aec3476cbb74961287fbc08e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38777 } }
I20260811 05:17:10.179615 29246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:17:10.179889 1142 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbe400f6aec3476cbb74961287fbc08e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cbe400f6aec3476cbb74961287fbc08e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe400f6aec3476cbb74961287fbc08e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38777 } } }
I20260811 05:17:10.179889 1148 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbe400f6aec3476cbb74961287fbc08e [sys.catalog]: SysCatalogTable state changed. Reason: New leader cbe400f6aec3476cbb74961287fbc08e. Latest consensus state: current_term: 1 leader_uuid: "cbe400f6aec3476cbb74961287fbc08e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe400f6aec3476cbb74961287fbc08e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38777 } } }
I20260811 05:17:10.180014 1142 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbe400f6aec3476cbb74961287fbc08e [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:10.180020 1148 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbe400f6aec3476cbb74961287fbc08e [sys.catalog]: This master's current role is: LEADER
W20260811 05:17:10.180526 1174 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cbe400f6aec3476cbb74961287fbc08e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:17:10.180572 1174 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:17:10.180657 1175 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:17:10.180864 1175 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:17:10.182999 1175 catalog_manager.cc:1347] Generated new cluster ID: b7bddbf32d9b4bf88a8e5099f3d30734
I20260811 05:17:10.183059 1175 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:17:10.185187 1139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:10.181802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41235 (local address 127.0.0.1:38777)
0811 05:17:10.181842 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:10.181844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:10.181922 (+ 78us) server_negotiation.cc:407] Connection header received
0811 05:17:10.181975 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:10.181975 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:10.181986 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:10.182047 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:10.182179 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:10.182785 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:10.183805 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:10.184142 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:10.184212 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:10.184945 (+ 733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:10.184947 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:10.184947 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:10.184952 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:10.184970 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:10.184975 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:10.185035 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:10.185099 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 05:17:10.185111 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 05:17:10.194401 1175 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:17:10.195098 1175 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:17:10.203775 1175 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cbe400f6aec3476cbb74961287fbc08e: Generated new TSK 0
I20260811 05:17:10.203991 1175 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:17:10.214890 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 1018
W20260811 05:17:10.227313 992 connection.cc:535] client connection to 127.0.0.1:38777 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38777): Connection reset by peer (error 104)
2026-08-11T05:17:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1514 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425159060450-29246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:17:10 dist-test-slave-f7th krb5kdc[1192](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:17:10 dist-test-slave-f7th krb5kdc[1192](info): set up 2 sockets
Aug 11 05:17:10 dist-test-slave-f7th krb5kdc[1192](info): commencing operation
krb5kdc: starting...
W20260811 05:17:11.430130 29246 mini_kdc.cc:121] Time spent starting KDC: real 1.189s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Aug 11 05:17:11 dist-test-slave-f7th krb5kdc[1192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:17:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:17:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 05:17:11.564656 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40055
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40055
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:11.648571 1209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:11.648828 1209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:11.648854 1209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:11.650090 1209 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:11.650144 1209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:11.650158 1209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:11.650170 1209 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:11.650182 1209 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:11.651788 1209 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40055
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1209
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:17:11.652056 1209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:11.652279 1209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:11.657249 1215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:11.657276 1214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:11.657249 1218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:11.659816 1209 server_base.cc:1034] running on GCE node
I20260811 05:17:11.660118 1209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:11.660405 1209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:11.661580 1209 hybrid_clock.cc:648] HybridClock initialized: now 1786425431661545 us; error 56 us; skew 500 ppm
Aug 11 05:17:11 dist-test-slave-f7th krb5kdc[1192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425431, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:11.664269 1209 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 05:17:11.664788 1209 webserver.cc:460] Webserver started at http://127.0.0.1:35783/ using document root <none> and password file <none>
I20260811 05:17:11.665040 1209 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:11.665091 1209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:11.665220 1209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:11.666280 1209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "5975673a88614ad293fd1d60b324f828"
format_stamp: "Formatted at 2026-08-11 05:17:11 on dist-test-slave-f7th"
server_key: "3debe7885d598bc23e50fc0c81125962"
server_key_iv: "6dda513dd06dd8c549a75f6a3e70392e"
server_key_version: "encryptionkey@0"
I20260811 05:17:11.666613 1209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "5975673a88614ad293fd1d60b324f828"
format_stamp: "Formatted at 2026-08-11 05:17:11 on dist-test-slave-f7th"
server_key: "3debe7885d598bc23e50fc0c81125962"
server_key_iv: "6dda513dd06dd8c549a75f6a3e70392e"
server_key_version: "encryptionkey@0"
I20260811 05:17:11.668015 1209 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 05:17:11.668829 1228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:11.669013 1209 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:17:11.669095 1209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "5975673a88614ad293fd1d60b324f828"
format_stamp: "Formatted at 2026-08-11 05:17:11 on dist-test-slave-f7th"
server_key: "3debe7885d598bc23e50fc0c81125962"
server_key_iv: "6dda513dd06dd8c549a75f6a3e70392e"
server_key_version: "encryptionkey@0"
I20260811 05:17:11.669160 1209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:11.672494 1209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:11.674713 1209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:11.674926 1209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:11.679045 1209 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40055
I20260811 05:17:11.679083 1333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40055 every 8 connection(s)
I20260811 05:17:11.679435 1209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:17:11.680437 1334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:11.680837 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 1209
I20260811 05:17:11.680949 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:17:11.681135 29246 external_mini_cluster.cc:1428] Setting key 17c1cda27773a1e8147ad626ab387348
I20260811 05:17:11.683653 1334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5975673a88614ad293fd1d60b324f828: Bootstrap starting.
I20260811 05:17:11.683861 1338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:11.681748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38099 (local address 127.0.0.1:40055)
0811 05:17:11.681950 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:11.681961 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:11.682000 (+ 39us) server_negotiation.cc:407] Connection header received
0811 05:17:11.682543 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:11.682552 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:11.682588 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:11.682753 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:11.683402 (+ 649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:11.683406 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:11.683407 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:11.683411 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:11.683461 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:11.683476 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:11.683553 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:11.683609 (+ 56us) server_negotiation.cc:299] Negotiation successful
0811 05:17:11.683651 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":87,"threads_started":1}
I20260811 05:17:11.684371 1334 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5975673a88614ad293fd1d60b324f828: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:11.684715 1334 log.cc:824] T 00000000000000000000000000000000 P 5975673a88614ad293fd1d60b324f828: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:11.685010 29246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:17:11.685611 1334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5975673a88614ad293fd1d60b324f828: No bootstrap required, opened a new log
I20260811 05:17:11.687676 1338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:11.686685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40655 (local address 127.0.0.1:40055)
0811 05:17:11.686732 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:11.686734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:11.686766 (+ 32us) server_negotiation.cc:407] Connection header received
0811 05:17:11.686830 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:11.686831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:11.686862 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:11.686909 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:11.687474 (+ 565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:11.687475 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:11.687476 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:11.687481 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:11.687497 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:11.687501 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:11.687549 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:11.687596 (+ 47us) server_negotiation.cc:299] Negotiation successful
0811 05:17:11.687610 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 05:17:11.687879 1334 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5975673a88614ad293fd1d60b324f828 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5975673a88614ad293fd1d60b324f828" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40055 } }
I20260811 05:17:11.688035 1334 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5975673a88614ad293fd1d60b324f828 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:11.688071 1334 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5975673a88614ad293fd1d60b324f828 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5975673a88614ad293fd1d60b324f828, State: Initialized, Role: FOLLOWER
I20260811 05:17:11.688194 1334 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5975673a88614ad293fd1d60b324f828 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5975673a88614ad293fd1d60b324f828" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40055 } }
I20260811 05:17:11.688278 1334 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5975673a88614ad293fd1d60b324f828 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:17:11.688311 1334 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5975673a88614ad293fd1d60b324f828 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:17:11.688364 1334 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5975673a88614ad293fd1d60b324f828 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:11.689195 1334 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5975673a88614ad293fd1d60b324f828 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5975673a88614ad293fd1d60b324f828" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40055 } }
I20260811 05:17:11.689318 1334 leader_election.cc:302] T 00000000000000000000000000000000 P 5975673a88614ad293fd1d60b324f828 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5975673a88614ad293fd1d60b324f828; no voters:
I20260811 05:17:11.689512 1334 leader_election.cc:288] T 00000000000000000000000000000000 P 5975673a88614ad293fd1d60b324f828 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:17:11.689586 1348 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5975673a88614ad293fd1d60b324f828 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:11.689769 1348 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5975673a88614ad293fd1d60b324f828 [term 1 LEADER]: Becoming Leader. State: Replica: 5975673a88614ad293fd1d60b324f828, State: Running, Role: LEADER
I20260811 05:17:11.689782 1334 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5975673a88614ad293fd1d60b324f828 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:11.689960 1348 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5975673a88614ad293fd1d60b324f828 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5975673a88614ad293fd1d60b324f828" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40055 } }
I20260811 05:17:11.690447 1351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5975673a88614ad293fd1d60b324f828 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5975673a88614ad293fd1d60b324f828" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5975673a88614ad293fd1d60b324f828" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40055 } } }
I20260811 05:17:11.690455 1352 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5975673a88614ad293fd1d60b324f828 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5975673a88614ad293fd1d60b324f828. Latest consensus state: current_term: 1 leader_uuid: "5975673a88614ad293fd1d60b324f828" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5975673a88614ad293fd1d60b324f828" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40055 } } }
I20260811 05:17:11.690551 1351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5975673a88614ad293fd1d60b324f828 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:11.690565 1352 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5975673a88614ad293fd1d60b324f828 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:11.691442 1368 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:17:11.692137 1368 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:17:11.693692 1368 catalog_manager.cc:1347] Generated new cluster ID: 9d19e76a72cd4ecda5ba0b3a59b95144
I20260811 05:17:11.693742 1368 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:17:11.701362 1368 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:17:11.701536 1368 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:17:11.704118 1368 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5975673a88614ad293fd1d60b324f828: Generated new TSK 0
I20260811 05:17:11.704298 1368 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:17:11.708211 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 1209
2026-08-11T05:17:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1492 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425159060450-29246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:17:11 dist-test-slave-f7th krb5kdc[1386](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:17:11 dist-test-slave-f7th krb5kdc[1386](info): set up 2 sockets
Aug 11 05:17:11 dist-test-slave-f7th krb5kdc[1386](info): commencing operation
krb5kdc: starting...
W20260811 05:17:12.911301 29246 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Aug 11 05:17:13 dist-test-slave-f7th krb5kdc[1386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425433, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:17:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:17:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 05:17:13.048213 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39063
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39063
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:13.133589 1413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:13.133847 1413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:13.133874 1413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:13.135205 1413 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:13.135259 1413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:13.135273 1413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:13.135285 1413 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:13.135296 1413 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:13.136947 1413 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39063
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1413
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:17:13.137181 1413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:13.137380 1413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:13.142391 1425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:13.142438 1419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:13.142478 1413 server_base.cc:1034] running on GCE node
W20260811 05:17:13.142422 1421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:13.142889 1413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:13.143188 1413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:13.144325 1413 hybrid_clock.cc:648] HybridClock initialized: now 1786425433144305 us; error 38 us; skew 500 ppm
Aug 11 05:17:13 dist-test-slave-f7th krb5kdc[1386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425433, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:13.152992 1413 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 05:17:13.153499 1413 webserver.cc:460] Webserver started at http://127.0.0.1:32955/ using document root <none> and password file <none>
I20260811 05:17:13.153718 1413 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:13.153767 1413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:13.153887 1413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:13.155066 1413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "f014e9fd290f4d0faa9eb8b658fd10e8"
format_stamp: "Formatted at 2026-08-11 05:17:13 on dist-test-slave-f7th"
server_key: "e64ab82253fffb470d92b749509259c8"
server_key_iv: "0456ce1df15b5e6cc80ceb4419ea814b"
server_key_version: "encryptionkey@0"
I20260811 05:17:13.155414 1413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "f014e9fd290f4d0faa9eb8b658fd10e8"
format_stamp: "Formatted at 2026-08-11 05:17:13 on dist-test-slave-f7th"
server_key: "e64ab82253fffb470d92b749509259c8"
server_key_iv: "0456ce1df15b5e6cc80ceb4419ea814b"
server_key_version: "encryptionkey@0"
I20260811 05:17:13.156899 1413 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 05:17:13.157747 1436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:13.157932 1413 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:17:13.158030 1413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "f014e9fd290f4d0faa9eb8b658fd10e8"
format_stamp: "Formatted at 2026-08-11 05:17:13 on dist-test-slave-f7th"
server_key: "e64ab82253fffb470d92b749509259c8"
server_key_iv: "0456ce1df15b5e6cc80ceb4419ea814b"
server_key_version: "encryptionkey@0"
I20260811 05:17:13.158097 1413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:13.174466 1413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:13.176527 1413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:13.176723 1413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:13.180765 1413 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39063
I20260811 05:17:13.180816 1535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39063 every 8 connection(s)
I20260811 05:17:13.181180 1413 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:17:13.182117 1536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:13.184592 1536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f014e9fd290f4d0faa9eb8b658fd10e8: Bootstrap starting.
I20260811 05:17:13.185003 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 1413
I20260811 05:17:13.185140 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:17:13.185328 29246 external_mini_cluster.cc:1428] Setting key cc60920879d5d16d27b89d637ab873e2
I20260811 05:17:13.185444 1536 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f014e9fd290f4d0faa9eb8b658fd10e8: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:13.185797 1536 log.cc:824] T 00000000000000000000000000000000 P f014e9fd290f4d0faa9eb8b658fd10e8: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:13.186765 1536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f014e9fd290f4d0faa9eb8b658fd10e8: No bootstrap required, opened a new log
I20260811 05:17:13.189196 1536 raft_consensus.cc:348] T 00000000000000000000000000000000 P f014e9fd290f4d0faa9eb8b658fd10e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f014e9fd290f4d0faa9eb8b658fd10e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39063 } }
I20260811 05:17:13.189357 1536 raft_consensus.cc:374] T 00000000000000000000000000000000 P f014e9fd290f4d0faa9eb8b658fd10e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:13.189380 1536 raft_consensus.cc:729] T 00000000000000000000000000000000 P f014e9fd290f4d0faa9eb8b658fd10e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f014e9fd290f4d0faa9eb8b658fd10e8, State: Initialized, Role: FOLLOWER
I20260811 05:17:13.189498 1536 consensus_queue.cc:260] T 00000000000000000000000000000000 P f014e9fd290f4d0faa9eb8b658fd10e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f014e9fd290f4d0faa9eb8b658fd10e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39063 } }
I20260811 05:17:13.189566 1536 raft_consensus.cc:388] T 00000000000000000000000000000000 P f014e9fd290f4d0faa9eb8b658fd10e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:17:13.189589 1536 raft_consensus.cc:482] T 00000000000000000000000000000000 P f014e9fd290f4d0faa9eb8b658fd10e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:17:13.189635 1536 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f014e9fd290f4d0faa9eb8b658fd10e8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:13.190374 1536 raft_consensus.cc:504] T 00000000000000000000000000000000 P f014e9fd290f4d0faa9eb8b658fd10e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f014e9fd290f4d0faa9eb8b658fd10e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39063 } }
I20260811 05:17:13.190490 1536 leader_election.cc:302] T 00000000000000000000000000000000 P f014e9fd290f4d0faa9eb8b658fd10e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f014e9fd290f4d0faa9eb8b658fd10e8; no voters:
I20260811 05:17:13.190670 1536 leader_election.cc:288] T 00000000000000000000000000000000 P f014e9fd290f4d0faa9eb8b658fd10e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:17:13.190768 1550 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f014e9fd290f4d0faa9eb8b658fd10e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:13.190963 1550 raft_consensus.cc:686] T 00000000000000000000000000000000 P f014e9fd290f4d0faa9eb8b658fd10e8 [term 1 LEADER]: Becoming Leader. State: Replica: f014e9fd290f4d0faa9eb8b658fd10e8, State: Running, Role: LEADER
I20260811 05:17:13.190968 1536 sys_catalog.cc:564] T 00000000000000000000000000000000 P f014e9fd290f4d0faa9eb8b658fd10e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:13.191120 1550 consensus_queue.cc:237] T 00000000000000000000000000000000 P f014e9fd290f4d0faa9eb8b658fd10e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f014e9fd290f4d0faa9eb8b658fd10e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39063 } }
I20260811 05:17:13.191511 1552 sys_catalog.cc:455] T 00000000000000000000000000000000 P f014e9fd290f4d0faa9eb8b658fd10e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f014e9fd290f4d0faa9eb8b658fd10e8. Latest consensus state: current_term: 1 leader_uuid: "f014e9fd290f4d0faa9eb8b658fd10e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f014e9fd290f4d0faa9eb8b658fd10e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39063 } } }
I20260811 05:17:13.191606 1552 sys_catalog.cc:458] T 00000000000000000000000000000000 P f014e9fd290f4d0faa9eb8b658fd10e8 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:13.191776 1551 sys_catalog.cc:455] T 00000000000000000000000000000000 P f014e9fd290f4d0faa9eb8b658fd10e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f014e9fd290f4d0faa9eb8b658fd10e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f014e9fd290f4d0faa9eb8b658fd10e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39063 } } }
I20260811 05:17:13.191839 1551 sys_catalog.cc:458] T 00000000000000000000000000000000 P f014e9fd290f4d0faa9eb8b658fd10e8 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:13.192196 1561 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:17:13.192764 1561 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 11 05:17:13 dist-test-slave-f7th krb5kdc[1386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425433, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 05:17:13.194581 1561 catalog_manager.cc:1347] Generated new cluster ID: 01f20442997e43edb82473571cd87345
I20260811 05:17:13.194636 1561 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:17:13.197665 1541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:13.185896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44205 (local address 127.0.0.1:39063)
0811 05:17:13.186145 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:13.186153 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:13.186181 (+ 28us) server_negotiation.cc:407] Connection header received
0811 05:17:13.186736 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:13.186746 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:13.186812 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:13.186990 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:13.187593 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:13.189066 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:13.189933 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:13.190566 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:13.190682 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:13.193860 (+ 3178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:13.193864 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:13.193865 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:13.193878 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:13.195789 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:13.196231 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:13.196234 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:13.196235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:13.196336 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:13.196536 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:13.196536 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:13.196536 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:13.196848 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:13.197010 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:13.197116 (+ 106us) server_negotiation.cc:299] Negotiation successful
0811 05:17:13.197255 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":140,"threads_started":1}
I20260811 05:17:13.198730 29246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:17:13.207307 1541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:13.200198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58673 (local address 127.0.0.1:39063)
0811 05:17:13.200253 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:13.200255 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:13.200487 (+ 232us) server_negotiation.cc:407] Connection header received
0811 05:17:13.200521 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:13.200522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:13.200533 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:13.200599 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:13.201200 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:13.201948 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:13.202908 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:13.203250 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:13.203314 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:13.204646 (+ 1332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:13.204648 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:13.204648 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:13.204656 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:13.206165 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:13.206548 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:13.206549 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:13.206550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:13.206613 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:13.206773 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:13.206774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:13.206774 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:13.206922 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:13.207015 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:13.207123 (+ 108us) server_negotiation.cc:299] Negotiation successful
0811 05:17:13.207206 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 05:17:13.223554 1561 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:17:13.224201 1561 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:17:13.234005 1561 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f014e9fd290f4d0faa9eb8b658fd10e8: Generated new TSK 0
I20260811 05:17:13.234203 1561 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:17:13.237753 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 1413
2026-08-11T05:17:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1533 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425159060450-29246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:17:13 dist-test-slave-f7th krb5kdc[1585](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:17:13 dist-test-slave-f7th krb5kdc[1585](info): set up 2 sockets
Aug 11 05:17:13 dist-test-slave-f7th krb5kdc[1585](info): commencing operation
krb5kdc: starting...
W20260811 05:17:14.450788 29246 mini_kdc.cc:121] Time spent starting KDC: real 1.183s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Aug 11 05:17:14 dist-test-slave-f7th krb5kdc[1585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:17:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:17:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 05:17:14.583917 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46059
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46059
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:14.667230 1602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:14.667488 1602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:14.667527 1602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:14.668761 1602 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:14.668807 1602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:14.668825 1602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:14.668841 1602 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:14.668857 1602 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:14.670467 1602 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46059
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46059
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:17:14.670723 1602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:14.670933 1602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:14.675794 1614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:14.675801 1611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:14.675864 1602 server_base.cc:1034] running on GCE node
W20260811 05:17:14.676050 1610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:14.676312 1602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:14.676605 1602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:14.677809 1602 hybrid_clock.cc:648] HybridClock initialized: now 1786425434677769 us; error 52 us; skew 500 ppm
Aug 11 05:17:14 dist-test-slave-f7th krb5kdc[1585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425434, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:14.680388 1602 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 05:17:14.680840 1602 webserver.cc:460] Webserver started at http://127.0.0.1:35473/ using document root <none> and password file <none>
I20260811 05:17:14.681056 1602 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:14.681097 1602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:14.681191 1602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:14.682216 1602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "aaa943a259384a228d50fbe3ca6ae325"
format_stamp: "Formatted at 2026-08-11 05:17:14 on dist-test-slave-f7th"
server_key: "b9e3a7c0db3e56128989442f29a88491"
server_key_iv: "d34802a94377f5c9f8f495eaba4554b0"
server_key_version: "encryptionkey@0"
I20260811 05:17:14.682531 1602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "aaa943a259384a228d50fbe3ca6ae325"
format_stamp: "Formatted at 2026-08-11 05:17:14 on dist-test-slave-f7th"
server_key: "b9e3a7c0db3e56128989442f29a88491"
server_key_iv: "d34802a94377f5c9f8f495eaba4554b0"
server_key_version: "encryptionkey@0"
I20260811 05:17:14.683753 1602 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 05:17:14.684466 1636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:14.684649 1602 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:17:14.684731 1602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "aaa943a259384a228d50fbe3ca6ae325"
format_stamp: "Formatted at 2026-08-11 05:17:14 on dist-test-slave-f7th"
server_key: "b9e3a7c0db3e56128989442f29a88491"
server_key_iv: "d34802a94377f5c9f8f495eaba4554b0"
server_key_version: "encryptionkey@0"
I20260811 05:17:14.684794 1602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:14.692675 1602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:14.694743 1602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:14.694886 1602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:14.698933 1602 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46059
I20260811 05:17:14.698998 1733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46059 every 8 connection(s)
I20260811 05:17:14.699328 1602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:17:14.699823 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 1602
I20260811 05:17:14.699965 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:17:14.700191 29246 external_mini_cluster.cc:1428] Setting key 93c98deaf1147c38a3a36e050382aebb
I20260811 05:17:14.700491 1734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:14.703902 1734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aaa943a259384a228d50fbe3ca6ae325: Bootstrap starting.
I20260811 05:17:14.704644 1734 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aaa943a259384a228d50fbe3ca6ae325: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:14.704999 1734 log.cc:824] T 00000000000000000000000000000000 P aaa943a259384a228d50fbe3ca6ae325: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:14.705861 1734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aaa943a259384a228d50fbe3ca6ae325: No bootstrap required, opened a new log
I20260811 05:17:14.706272 1739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:14.700835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38925 (local address 127.0.0.1:46059)
0811 05:17:14.701062 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:14.701069 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:14.701096 (+ 27us) server_negotiation.cc:407] Connection header received
0811 05:17:14.701717 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:14.701726 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:14.701781 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:14.701914 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:14.702257 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:14.703175 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:14.704254 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:14.704765 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:14.704881 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:14.705615 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:14.705619 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:14.705619 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:14.705781 (+ 162us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:14.705812 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:14.705823 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:14.705877 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:14.705987 (+ 110us) server_negotiation.cc:299] Negotiation successful
0811 05:17:14.706047 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":74,"threads_started":1}
I20260811 05:17:14.707494 1734 raft_consensus.cc:348] T 00000000000000000000000000000000 P aaa943a259384a228d50fbe3ca6ae325 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aaa943a259384a228d50fbe3ca6ae325" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46059 } }
I20260811 05:17:14.707633 1734 raft_consensus.cc:374] T 00000000000000000000000000000000 P aaa943a259384a228d50fbe3ca6ae325 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:14.707667 1734 raft_consensus.cc:729] T 00000000000000000000000000000000 P aaa943a259384a228d50fbe3ca6ae325 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aaa943a259384a228d50fbe3ca6ae325, State: Initialized, Role: FOLLOWER
I20260811 05:17:14.707816 1734 consensus_queue.cc:260] T 00000000000000000000000000000000 P aaa943a259384a228d50fbe3ca6ae325 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aaa943a259384a228d50fbe3ca6ae325" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46059 } }
I20260811 05:17:14.707913 1734 raft_consensus.cc:388] T 00000000000000000000000000000000 P aaa943a259384a228d50fbe3ca6ae325 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:17:14.707944 1734 raft_consensus.cc:482] T 00000000000000000000000000000000 P aaa943a259384a228d50fbe3ca6ae325 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:17:14.707975 1734 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aaa943a259384a228d50fbe3ca6ae325 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:14.707965 29246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:17:14.708540 1734 raft_consensus.cc:504] T 00000000000000000000000000000000 P aaa943a259384a228d50fbe3ca6ae325 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aaa943a259384a228d50fbe3ca6ae325" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46059 } }
I20260811 05:17:14.708663 1734 leader_election.cc:302] T 00000000000000000000000000000000 P aaa943a259384a228d50fbe3ca6ae325 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aaa943a259384a228d50fbe3ca6ae325; no voters:
I20260811 05:17:14.708829 1734 leader_election.cc:288] T 00000000000000000000000000000000 P aaa943a259384a228d50fbe3ca6ae325 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:17:14.708895 1742 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aaa943a259384a228d50fbe3ca6ae325 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:14.709095 1734 sys_catalog.cc:564] T 00000000000000000000000000000000 P aaa943a259384a228d50fbe3ca6ae325 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:14.709120 1742 raft_consensus.cc:686] T 00000000000000000000000000000000 P aaa943a259384a228d50fbe3ca6ae325 [term 1 LEADER]: Becoming Leader. State: Replica: aaa943a259384a228d50fbe3ca6ae325, State: Running, Role: LEADER
I20260811 05:17:14.709268 1742 consensus_queue.cc:237] T 00000000000000000000000000000000 P aaa943a259384a228d50fbe3ca6ae325 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aaa943a259384a228d50fbe3ca6ae325" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46059 } }
I20260811 05:17:14.709666 1753 sys_catalog.cc:455] T 00000000000000000000000000000000 P aaa943a259384a228d50fbe3ca6ae325 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aaa943a259384a228d50fbe3ca6ae325. Latest consensus state: current_term: 1 leader_uuid: "aaa943a259384a228d50fbe3ca6ae325" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aaa943a259384a228d50fbe3ca6ae325" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46059 } } }
I20260811 05:17:14.709756 1753 sys_catalog.cc:458] T 00000000000000000000000000000000 P aaa943a259384a228d50fbe3ca6ae325 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:14.710103 1747 sys_catalog.cc:455] T 00000000000000000000000000000000 P aaa943a259384a228d50fbe3ca6ae325 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aaa943a259384a228d50fbe3ca6ae325" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aaa943a259384a228d50fbe3ca6ae325" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46059 } } }
I20260811 05:17:14.710165 1747 sys_catalog.cc:458] T 00000000000000000000000000000000 P aaa943a259384a228d50fbe3ca6ae325 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:14.710628 1759 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:17:14.711288 1759 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:17:14.712978 1739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:14.709471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32847 (local address 127.0.0.1:46059)
0811 05:17:14.709518 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:14.709520 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:14.709638 (+ 118us) server_negotiation.cc:407] Connection header received
0811 05:17:14.709736 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:14.709737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:14.709748 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:14.709778 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:14.710032 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:14.710701 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:14.711705 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:14.711990 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:14.712026 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:14.712737 (+ 711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:14.712738 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:14.712739 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:14.712744 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:14.712760 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:14.712765 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:14.712791 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:14.712881 (+ 90us) server_negotiation.cc:299] Negotiation successful
0811 05:17:14.712904 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 05:17:14.713523 1759 catalog_manager.cc:1347] Generated new cluster ID: f9227b7724334a70a0a70b5193f97d44
I20260811 05:17:14.713572 1759 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:17:14.721935 1759 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:17:14.722647 1759 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:17:14.728212 1759 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aaa943a259384a228d50fbe3ca6ae325: Generated new TSK 0
I20260811 05:17:14.728454 1759 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:17:14.738684 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 1602
2026-08-11T05:17:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1496 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425159060450-29246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:17:14 dist-test-slave-f7th krb5kdc[1783](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:17:14 dist-test-slave-f7th krb5kdc[1783](info): set up 2 sockets
Aug 11 05:17:14 dist-test-slave-f7th krb5kdc[1783](info): commencing operation
krb5kdc: starting...
W20260811 05:17:15.948385 29246 mini_kdc.cc:121] Time spent starting KDC: real 1.184s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Aug 11 05:17:16 dist-test-slave-f7th krb5kdc[1783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425436, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:17:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:17:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 05:17:16.085191 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42675
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42675
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:16.170588 1804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:16.170832 1804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:16.170866 1804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:16.172089 1804 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:16.172139 1804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:16.172158 1804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:16.172173 1804 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:16.172189 1804 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:16.173818 1804 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42675
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1804
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:17:16.174098 1804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:16.174309 1804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:16.179183 1813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:16.179214 1811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:16.179189 1819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:16.179606 1804 server_base.cc:1034] running on GCE node
I20260811 05:17:16.179854 1804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:16.180163 1804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:16.181301 1804 hybrid_clock.cc:648] HybridClock initialized: now 1786425436181269 us; error 52 us; skew 500 ppm
Aug 11 05:17:16 dist-test-slave-f7th krb5kdc[1783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425436, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:16.183801 1804 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 05:17:16.184329 1804 webserver.cc:460] Webserver started at http://127.0.0.1:39755/ using document root <none> and password file <none>
I20260811 05:17:16.184556 1804 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:16.184608 1804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:16.184785 1804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:16.185887 1804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "f8e284f69a704426a6617209416c81b8"
format_stamp: "Formatted at 2026-08-11 05:17:16 on dist-test-slave-f7th"
server_key: "560c8d390fba9299a19566475b5c9115"
server_key_iv: "1c1495ad2b5fafcf6a0d3f5fa1c17607"
server_key_version: "encryptionkey@0"
I20260811 05:17:16.186252 1804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "f8e284f69a704426a6617209416c81b8"
format_stamp: "Formatted at 2026-08-11 05:17:16 on dist-test-slave-f7th"
server_key: "560c8d390fba9299a19566475b5c9115"
server_key_iv: "1c1495ad2b5fafcf6a0d3f5fa1c17607"
server_key_version: "encryptionkey@0"
I20260811 05:17:16.187697 1804 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 05:17:16.188565 1829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:16.188788 1804 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:17:16.188891 1804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "f8e284f69a704426a6617209416c81b8"
format_stamp: "Formatted at 2026-08-11 05:17:16 on dist-test-slave-f7th"
server_key: "560c8d390fba9299a19566475b5c9115"
server_key_iv: "1c1495ad2b5fafcf6a0d3f5fa1c17607"
server_key_version: "encryptionkey@0"
I20260811 05:17:16.188961 1804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:16.192360 1804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:16.194255 1804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:16.194399 1804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:16.198534 1804 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42675
I20260811 05:17:16.198572 1922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42675 every 8 connection(s)
I20260811 05:17:16.198921 1804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:17:16.199834 1925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:16.201326 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 1804
I20260811 05:17:16.201447 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:17:16.201658 29246 external_mini_cluster.cc:1428] Setting key 7c26a7132590b8b38bbf4c6d7176bb3f
I20260811 05:17:16.202863 1925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8e284f69a704426a6617209416c81b8: Bootstrap starting.
I20260811 05:17:16.203650 1925 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8e284f69a704426a6617209416c81b8: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:16.204011 1925 log.cc:824] T 00000000000000000000000000000000 P f8e284f69a704426a6617209416c81b8: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:16.204524 1931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:16.202260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48375 (local address 127.0.0.1:42675)
0811 05:17:16.202494 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:16.202504 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:16.202539 (+ 35us) server_negotiation.cc:407] Connection header received
0811 05:17:16.203082 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:16.203092 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:16.203128 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:16.203302 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:16.204056 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:16.204060 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:16.204060 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:16.204073 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:16.204127 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:16.204143 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:16.204243 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:16.204300 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 05:17:16.204346 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":102,"threads_started":1}
I20260811 05:17:16.204825 1925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8e284f69a704426a6617209416c81b8: No bootstrap required, opened a new log
I20260811 05:17:16.205672 29246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:17:16.206899 1925 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8e284f69a704426a6617209416c81b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e284f69a704426a6617209416c81b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42675 } }
I20260811 05:17:16.207065 1925 raft_consensus.cc:374] T 00000000000000000000000000000000 P f8e284f69a704426a6617209416c81b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:16.207088 1925 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8e284f69a704426a6617209416c81b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8e284f69a704426a6617209416c81b8, State: Initialized, Role: FOLLOWER
I20260811 05:17:16.207262 1925 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8e284f69a704426a6617209416c81b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e284f69a704426a6617209416c81b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42675 } }
I20260811 05:17:16.207363 1925 raft_consensus.cc:388] T 00000000000000000000000000000000 P f8e284f69a704426a6617209416c81b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:17:16.207401 1925 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8e284f69a704426a6617209416c81b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:17:16.207477 1925 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8e284f69a704426a6617209416c81b8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:16.208230 1925 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8e284f69a704426a6617209416c81b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e284f69a704426a6617209416c81b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42675 } }
I20260811 05:17:16.208298 1931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:16.207231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41301 (local address 127.0.0.1:42675)
0811 05:17:16.207278 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:16.207280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:16.207306 (+ 26us) server_negotiation.cc:407] Connection header received
0811 05:17:16.207388 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:16.207389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:16.207394 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:16.207440 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:16.208047 (+ 607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:16.208049 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:16.208049 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:16.208053 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:16.208068 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:16.208073 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:16.208121 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:16.208177 (+ 56us) server_negotiation.cc:299] Negotiation successful
0811 05:17:16.208192 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 05:17:16.208384 1925 leader_election.cc:302] T 00000000000000000000000000000000 P f8e284f69a704426a6617209416c81b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f8e284f69a704426a6617209416c81b8; no voters:
I20260811 05:17:16.208597 1925 leader_election.cc:288] T 00000000000000000000000000000000 P f8e284f69a704426a6617209416c81b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:17:16.208673 1944 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8e284f69a704426a6617209416c81b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:16.208828 1944 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8e284f69a704426a6617209416c81b8 [term 1 LEADER]: Becoming Leader. State: Replica: f8e284f69a704426a6617209416c81b8, State: Running, Role: LEADER
I20260811 05:17:16.208945 1925 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8e284f69a704426a6617209416c81b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:16.208982 1944 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8e284f69a704426a6617209416c81b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e284f69a704426a6617209416c81b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42675 } }
I20260811 05:17:16.209333 1944 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8e284f69a704426a6617209416c81b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8e284f69a704426a6617209416c81b8. Latest consensus state: current_term: 1 leader_uuid: "f8e284f69a704426a6617209416c81b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e284f69a704426a6617209416c81b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42675 } } }
I20260811 05:17:16.209460 1944 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8e284f69a704426a6617209416c81b8 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:16.209362 1947 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8e284f69a704426a6617209416c81b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f8e284f69a704426a6617209416c81b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8e284f69a704426a6617209416c81b8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42675 } } }
I20260811 05:17:16.209594 1947 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8e284f69a704426a6617209416c81b8 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:16.209738 1960 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:17:16.210234 1960 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:17:16.211861 1960 catalog_manager.cc:1347] Generated new cluster ID: 7107124f972848708396656bc1ca04d4
I20260811 05:17:16.211920 1960 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:17:16.231837 1960 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:17:16.232070 1960 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:17:16.238598 1960 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f8e284f69a704426a6617209416c81b8: Generated new TSK 0
I20260811 05:17:16.238847 1960 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:17:16.245642 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 1804
2026-08-11T05:17:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1503 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425159060450-29246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:17:16 dist-test-slave-f7th krb5kdc[1979](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:17:16 dist-test-slave-f7th krb5kdc[1979](info): set up 2 sockets
Aug 11 05:17:16 dist-test-slave-f7th krb5kdc[1979](info): commencing operation
krb5kdc: starting...
W20260811 05:17:17.478286 29246 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Aug 11 05:17:17 dist-test-slave-f7th krb5kdc[1979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:17:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:17:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 05:17:17.612851 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39499
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39499
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:17.700598 2007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:17.700881 2007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:17.700920 2007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:17.702248 2007 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:17.702309 2007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:17.702329 2007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:17.702346 2007 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:17.702363 2007 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:17.704051 2007 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39499
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:17:17.704357 2007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:17.704594 2007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:17.710011 2018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:17.710000 2015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:17.710131 2007 server_base.cc:1034] running on GCE node
W20260811 05:17:17.710000 2014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:17.710520 2007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:17.710861 2007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:17.712082 2007 hybrid_clock.cc:648] HybridClock initialized: now 1786425437712061 us; error 30 us; skew 500 ppm
Aug 11 05:17:17 dist-test-slave-f7th krb5kdc[1979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425437, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:17.715375 2007 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 05:17:17.715894 2007 webserver.cc:460] Webserver started at http://127.0.0.1:37505/ using document root <none> and password file <none>
I20260811 05:17:17.716141 2007 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:17.716192 2007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:17.716311 2007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:17.717499 2007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "1d60734faf7d4645b576588b14876641"
format_stamp: "Formatted at 2026-08-11 05:17:17 on dist-test-slave-f7th"
server_key: "870c42a4c356975e082ef4a95ae5c3e2"
server_key_iv: "00caabb5ab61e2d2c36a7b1f74d0297f"
server_key_version: "encryptionkey@0"
I20260811 05:17:17.717886 2007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "1d60734faf7d4645b576588b14876641"
format_stamp: "Formatted at 2026-08-11 05:17:17 on dist-test-slave-f7th"
server_key: "870c42a4c356975e082ef4a95ae5c3e2"
server_key_iv: "00caabb5ab61e2d2c36a7b1f74d0297f"
server_key_version: "encryptionkey@0"
I20260811 05:17:17.719333 2007 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 05:17:17.720152 2028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:17.720407 2007 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:17:17.720497 2007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "1d60734faf7d4645b576588b14876641"
format_stamp: "Formatted at 2026-08-11 05:17:17 on dist-test-slave-f7th"
server_key: "870c42a4c356975e082ef4a95ae5c3e2"
server_key_iv: "00caabb5ab61e2d2c36a7b1f74d0297f"
server_key_version: "encryptionkey@0"
I20260811 05:17:17.720561 2007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:17.735247 2007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:17.737517 2007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:17.737644 2007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:17.741592 2007 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39499
I20260811 05:17:17.741649 2136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39499 every 8 connection(s)
I20260811 05:17:17.741986 2007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:17:17.742864 2137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:17.745326 2137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d60734faf7d4645b576588b14876641: Bootstrap starting.
I20260811 05:17:17.745921 2137 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d60734faf7d4645b576588b14876641: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:17.746188 2137 log.cc:824] T 00000000000000000000000000000000 P 1d60734faf7d4645b576588b14876641: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:17.746946 2137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d60734faf7d4645b576588b14876641: No bootstrap required, opened a new log
I20260811 05:17:17.748670 2137 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d60734faf7d4645b576588b14876641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d60734faf7d4645b576588b14876641" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39499 } }
I20260811 05:17:17.748927 2137 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d60734faf7d4645b576588b14876641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:17.749001 2137 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d60734faf7d4645b576588b14876641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d60734faf7d4645b576588b14876641, State: Initialized, Role: FOLLOWER
I20260811 05:17:17.749173 2137 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d60734faf7d4645b576588b14876641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d60734faf7d4645b576588b14876641" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39499 } }
I20260811 05:17:17.749254 2137 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d60734faf7d4645b576588b14876641 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:17:17.749276 2137 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d60734faf7d4645b576588b14876641 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:17:17.749310 2137 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d60734faf7d4645b576588b14876641 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:17.749676 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 2007
I20260811 05:17:17.749761 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:17:17.749938 2137 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d60734faf7d4645b576588b14876641 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d60734faf7d4645b576588b14876641" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39499 } }
I20260811 05:17:17.749977 29246 external_mini_cluster.cc:1428] Setting key ad26688ee97cbd742204de8370cfe9c8
I20260811 05:17:17.750042 2137 leader_election.cc:302] T 00000000000000000000000000000000 P 1d60734faf7d4645b576588b14876641 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d60734faf7d4645b576588b14876641; no voters:
I20260811 05:17:17.750247 2137 leader_election.cc:288] T 00000000000000000000000000000000 P 1d60734faf7d4645b576588b14876641 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:17:17.750278 2141 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d60734faf7d4645b576588b14876641 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:17.750409 2141 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d60734faf7d4645b576588b14876641 [term 1 LEADER]: Becoming Leader. State: Replica: 1d60734faf7d4645b576588b14876641, State: Running, Role: LEADER
I20260811 05:17:17.750542 2137 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d60734faf7d4645b576588b14876641 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:17.750561 2141 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d60734faf7d4645b576588b14876641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d60734faf7d4645b576588b14876641" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39499 } }
I20260811 05:17:17.751128 2143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d60734faf7d4645b576588b14876641 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1d60734faf7d4645b576588b14876641" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d60734faf7d4645b576588b14876641" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39499 } } }
I20260811 05:17:17.751268 2143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d60734faf7d4645b576588b14876641 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:17.751502 2145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d60734faf7d4645b576588b14876641 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d60734faf7d4645b576588b14876641. Latest consensus state: current_term: 1 leader_uuid: "1d60734faf7d4645b576588b14876641" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d60734faf7d4645b576588b14876641" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39499 } } }
I20260811 05:17:17.751626 2145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d60734faf7d4645b576588b14876641 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:17.751680 2160 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:17:17.752264 2160 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:17:17.754407 2160 catalog_manager.cc:1347] Generated new cluster ID: 3fc12eab55564fe99949627f4e6c024f
I20260811 05:17:17.754477 2160 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 05:17:17 dist-test-slave-f7th krb5kdc[1979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 05:17:17.761710 2155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:17.750789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:44129 (local address 127.0.0.1:39499)
0811 05:17:17.751011 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:17.751023 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:17.751056 (+ 33us) server_negotiation.cc:407] Connection header received
0811 05:17:17.751186 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:17.751193 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:17.751260 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:17.751422 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:17.752241 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:17.753894 (+ 1653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:17.754795 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:17.755227 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:17.755331 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:17.757922 (+ 2591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:17.757927 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:17.757928 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:17.757943 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:17.760042 (+ 2099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:17.760464 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:17.760468 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:17.760468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:17.760579 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:17.760843 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:17.760844 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:17.760845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:17.761133 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:17.761316 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:17.761406 (+ 90us) server_negotiation.cc:299] Negotiation successful
0811 05:17:17.761517 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":88,"threads_started":1}
I20260811 05:17:17.762717 29246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:17:17.771049 2155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:17.764986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:58399 (local address 127.0.0.1:39499)
0811 05:17:17.765031 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:17.765033 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:17.765248 (+ 215us) server_negotiation.cc:407] Connection header received
0811 05:17:17.765307 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:17.765308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:17.765320 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:17.765360 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:17.765885 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:17.766381 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:17.767135 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:17.767353 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:17.767406 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:17.768574 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:17.768575 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:17.768575 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:17.768583 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:17.769923 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:17.770303 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:17.770305 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:17.770305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:17.770362 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:17.770562 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:17.770563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:17.770563 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:17.770679 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:17.770785 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:17.770897 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 05:17:17.770959 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 05:17:17.801707 2160 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:17:17.802457 2160 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:17:17.816423 2160 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d60734faf7d4645b576588b14876641: Generated new TSK 0
I20260811 05:17:17.816708 2160 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:17:17.821439 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 2007
W20260811 05:17:17.834167 1977 connection.cc:535] client connection to 127.0.0.1:39499 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39499): Connection reset by peer (error 104)
2026-08-11T05:17:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1580 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425159060450-29246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:17:17 dist-test-slave-f7th krb5kdc[2194](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:17:17 dist-test-slave-f7th krb5kdc[2194](info): set up 2 sockets
Aug 11 05:17:17 dist-test-slave-f7th krb5kdc[2194](info): commencing operation
krb5kdc: starting...
W20260811 05:17:19.041006 29246 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Aug 11 05:17:19 dist-test-slave-f7th krb5kdc[2194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:17:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:17:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 05:17:19.171871 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36769
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36769
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:19.261943 2222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:19.262188 2222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:19.262223 2222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:19.263484 2222 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:19.263537 2222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:19.263564 2222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:19.263582 2222 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:19.263608 2222 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:19.265316 2222 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36769
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:17:19.265599 2222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:19.265825 2222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:19.270982 2232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:19.270999 2234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:19.271163 2238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:19.271348 2222 server_base.cc:1034] running on GCE node
I20260811 05:17:19.271606 2222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:19.271903 2222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:19.273175 2222 hybrid_clock.cc:648] HybridClock initialized: now 1786425439273149 us; error 52 us; skew 500 ppm
Aug 11 05:17:19 dist-test-slave-f7th krb5kdc[2194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425439, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:19.280592 2222 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 05:17:19.281188 2222 webserver.cc:460] Webserver started at http://127.0.0.1:36381/ using document root <none> and password file <none>
I20260811 05:17:19.281458 2222 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:19.281524 2222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:19.281642 2222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:19.282760 2222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "04894614244e450d8e9b0dd652b8ea40"
format_stamp: "Formatted at 2026-08-11 05:17:19 on dist-test-slave-f7th"
server_key: "817f271f7453550be454d4b0abbc5ae4"
server_key_iv: "637b6f83aed38b3a0ec2b9ee2c735f71"
server_key_version: "encryptionkey@0"
I20260811 05:17:19.283134 2222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "04894614244e450d8e9b0dd652b8ea40"
format_stamp: "Formatted at 2026-08-11 05:17:19 on dist-test-slave-f7th"
server_key: "817f271f7453550be454d4b0abbc5ae4"
server_key_iv: "637b6f83aed38b3a0ec2b9ee2c735f71"
server_key_version: "encryptionkey@0"
I20260811 05:17:19.284868 2222 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260811 05:17:19.285822 2247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:19.286048 2222 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:17:19.286150 2222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "04894614244e450d8e9b0dd652b8ea40"
format_stamp: "Formatted at 2026-08-11 05:17:19 on dist-test-slave-f7th"
server_key: "817f271f7453550be454d4b0abbc5ae4"
server_key_iv: "637b6f83aed38b3a0ec2b9ee2c735f71"
server_key_version: "encryptionkey@0"
I20260811 05:17:19.286213 2222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:19.299469 2222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:19.301741 2222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:19.301867 2222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:19.306103 2222 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36769
I20260811 05:17:19.306226 2336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36769 every 8 connection(s)
I20260811 05:17:19.306533 2222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:17:19.307487 2337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:19.308369 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 2222
I20260811 05:17:19.308527 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:17:19.308779 29246 external_mini_cluster.cc:1428] Setting key ab550d355e797f21ce7efe9a819670ce
I20260811 05:17:19.310336 2337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04894614244e450d8e9b0dd652b8ea40: Bootstrap starting.
I20260811 05:17:19.310937 2337 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04894614244e450d8e9b0dd652b8ea40: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:19.311215 2337 log.cc:824] T 00000000000000000000000000000000 P 04894614244e450d8e9b0dd652b8ea40: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:19.312088 2337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04894614244e450d8e9b0dd652b8ea40: No bootstrap required, opened a new log
I20260811 05:17:19.314337 2337 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04894614244e450d8e9b0dd652b8ea40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04894614244e450d8e9b0dd652b8ea40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36769 } }
I20260811 05:17:19.314512 2337 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04894614244e450d8e9b0dd652b8ea40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:19.314574 2337 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04894614244e450d8e9b0dd652b8ea40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04894614244e450d8e9b0dd652b8ea40, State: Initialized, Role: FOLLOWER
I20260811 05:17:19.314736 2341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:19.309352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:60891 (local address 127.0.0.1:36769)
0811 05:17:19.309557 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:19.309566 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:19.309592 (+ 26us) server_negotiation.cc:407] Connection header received
0811 05:17:19.310109 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:19.310115 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:19.310167 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:19.310298 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:19.310528 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:19.311616 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:19.312757 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:19.313206 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:19.313291 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:19.314005 (+ 714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:19.314009 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:19.314009 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:19.314165 (+ 156us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:19.314195 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:19.314206 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:19.314269 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:19.314378 (+ 109us) server_negotiation.cc:299] Negotiation successful
0811 05:17:19.314403 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":97,"threads_started":1}
I20260811 05:17:19.314715 2337 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04894614244e450d8e9b0dd652b8ea40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04894614244e450d8e9b0dd652b8ea40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36769 } }
I20260811 05:17:19.314808 2337 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04894614244e450d8e9b0dd652b8ea40 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:17:19.314846 2337 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04894614244e450d8e9b0dd652b8ea40 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:17:19.314894 2337 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04894614244e450d8e9b0dd652b8ea40 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:19.315492 2337 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04894614244e450d8e9b0dd652b8ea40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04894614244e450d8e9b0dd652b8ea40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36769 } }
I20260811 05:17:19.315596 2337 leader_election.cc:302] T 00000000000000000000000000000000 P 04894614244e450d8e9b0dd652b8ea40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04894614244e450d8e9b0dd652b8ea40; no voters:
I20260811 05:17:19.315726 29246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:17:19.315764 2337 leader_election.cc:288] T 00000000000000000000000000000000 P 04894614244e450d8e9b0dd652b8ea40 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:17:19.315798 2349 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04894614244e450d8e9b0dd652b8ea40 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:19.316008 2349 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04894614244e450d8e9b0dd652b8ea40 [term 1 LEADER]: Becoming Leader. State: Replica: 04894614244e450d8e9b0dd652b8ea40, State: Running, Role: LEADER
I20260811 05:17:19.316015 2337 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04894614244e450d8e9b0dd652b8ea40 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:19.316188 2349 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04894614244e450d8e9b0dd652b8ea40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04894614244e450d8e9b0dd652b8ea40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36769 } }
I20260811 05:17:19.316551 2350 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04894614244e450d8e9b0dd652b8ea40 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04894614244e450d8e9b0dd652b8ea40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04894614244e450d8e9b0dd652b8ea40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36769 } } }
I20260811 05:17:19.316673 2350 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04894614244e450d8e9b0dd652b8ea40 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:19.316550 2351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04894614244e450d8e9b0dd652b8ea40 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04894614244e450d8e9b0dd652b8ea40. Latest consensus state: current_term: 1 leader_uuid: "04894614244e450d8e9b0dd652b8ea40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04894614244e450d8e9b0dd652b8ea40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36769 } } }
I20260811 05:17:19.316900 2351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04894614244e450d8e9b0dd652b8ea40 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:19.317473 2361 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:17:19.318073 2361 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:17:19.319761 2361 catalog_manager.cc:1347] Generated new cluster ID: 0ec5ca9ad1f94c18bf1a209b639a09d4
I20260811 05:17:19.319814 2361 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:17:19.321156 2341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:19.317289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:46671 (local address 127.0.0.1:36769)
0811 05:17:19.317323 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:19.317324 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:19.317792 (+ 468us) server_negotiation.cc:407] Connection header received
0811 05:17:19.317868 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:19.317868 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:19.317880 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:19.317939 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:19.318139 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:19.318828 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:19.319711 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:19.320045 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:19.320118 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:19.320915 (+ 797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:19.320917 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:19.320918 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:19.320923 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:19.320940 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:19.320945 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:19.321009 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:19.321064 (+ 55us) server_negotiation.cc:299] Negotiation successful
0811 05:17:19.321076 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":15}
I20260811 05:17:19.335268 2361 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:17:19.336084 2361 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:17:19.352725 2361 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04894614244e450d8e9b0dd652b8ea40: Generated new TSK 0
I20260811 05:17:19.352981 2361 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:17:19.360476 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 2222
W20260811 05:17:19.368103 2190 connection.cc:535] client connection to 127.0.0.1:36769 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36769): Connection reset by peer (error 104)
2026-08-11T05:17:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1533 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425159060450-29246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:17:19 dist-test-slave-f7th krb5kdc[2389](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:17:19 dist-test-slave-f7th krb5kdc[2389](info): set up 2 sockets
Aug 11 05:17:19 dist-test-slave-f7th krb5kdc[2389](info): commencing operation
krb5kdc: starting...
W20260811 05:17:20.569025 29246 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Aug 11 05:17:20 dist-test-slave-f7th krb5kdc[2389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425440, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:17:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:17:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 05:17:20.704511 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40079
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40079
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:20.791344 2412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:20.791597 2412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:20.791633 2412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:20.792999 2412 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:20.793057 2412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:20.793076 2412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:20.793093 2412 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:20.793109 2412 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:20.794806 2412 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40079
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:17:20.795269 2412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:20.795670 2412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:20.801155 2420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:20.801154 2423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:20.801246 2421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:20.801568 2412 server_base.cc:1034] running on GCE node
I20260811 05:17:20.801836 2412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:20.802173 2412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:20.803351 2412 hybrid_clock.cc:648] HybridClock initialized: now 1786425440803332 us; error 30 us; skew 500 ppm
Aug 11 05:17:20 dist-test-slave-f7th krb5kdc[2389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425440, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:20.806522 2412 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 05:17:20.807092 2412 webserver.cc:460] Webserver started at http://127.0.0.1:45933/ using document root <none> and password file <none>
I20260811 05:17:20.807338 2412 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:20.807392 2412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:20.807531 2412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:20.808705 2412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "17bdf775d0b1467c9401049b0fbae242"
format_stamp: "Formatted at 2026-08-11 05:17:20 on dist-test-slave-f7th"
server_key: "d62b28928b02eeb6ac1d901370e86247"
server_key_iv: "036615cc85f4f4757b5648d0b2192b3c"
server_key_version: "encryptionkey@0"
I20260811 05:17:20.809084 2412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "17bdf775d0b1467c9401049b0fbae242"
format_stamp: "Formatted at 2026-08-11 05:17:20 on dist-test-slave-f7th"
server_key: "d62b28928b02eeb6ac1d901370e86247"
server_key_iv: "036615cc85f4f4757b5648d0b2192b3c"
server_key_version: "encryptionkey@0"
I20260811 05:17:20.810672 2412 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 05:17:20.811549 2436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:20.811772 2412 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 05:17:20.811883 2412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "17bdf775d0b1467c9401049b0fbae242"
format_stamp: "Formatted at 2026-08-11 05:17:20 on dist-test-slave-f7th"
server_key: "d62b28928b02eeb6ac1d901370e86247"
server_key_iv: "036615cc85f4f4757b5648d0b2192b3c"
server_key_version: "encryptionkey@0"
I20260811 05:17:20.811959 2412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:20.815747 2412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:20.818207 2412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:20.818365 2412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:20.823303 2412 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40079
I20260811 05:17:20.823369 2541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40079 every 8 connection(s)
I20260811 05:17:20.823800 2412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:17:20.824884 2542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:20.827414 2542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17bdf775d0b1467c9401049b0fbae242: Bootstrap starting.
I20260811 05:17:20.828063 2542 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17bdf775d0b1467c9401049b0fbae242: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:20.828352 2542 log.cc:824] T 00000000000000000000000000000000 P 17bdf775d0b1467c9401049b0fbae242: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:20.829304 2542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17bdf775d0b1467c9401049b0fbae242: No bootstrap required, opened a new log
I20260811 05:17:20.830760 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 2412
I20260811 05:17:20.830868 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:17:20.831311 29246 external_mini_cluster.cc:1428] Setting key fc0102b8a128c49c8637ba395ac2486d
I20260811 05:17:20.831487 2542 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17bdf775d0b1467c9401049b0fbae242 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17bdf775d0b1467c9401049b0fbae242" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40079 } }
I20260811 05:17:20.831723 2542 raft_consensus.cc:374] T 00000000000000000000000000000000 P 17bdf775d0b1467c9401049b0fbae242 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:20.831768 2542 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17bdf775d0b1467c9401049b0fbae242 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17bdf775d0b1467c9401049b0fbae242, State: Initialized, Role: FOLLOWER
I20260811 05:17:20.832091 2542 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17bdf775d0b1467c9401049b0fbae242 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17bdf775d0b1467c9401049b0fbae242" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40079 } }
I20260811 05:17:20.832273 2542 raft_consensus.cc:388] T 00000000000000000000000000000000 P 17bdf775d0b1467c9401049b0fbae242 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:17:20.832356 2542 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17bdf775d0b1467c9401049b0fbae242 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:17:20.832476 2542 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17bdf775d0b1467c9401049b0fbae242 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:20.833793 2542 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17bdf775d0b1467c9401049b0fbae242 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17bdf775d0b1467c9401049b0fbae242" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40079 } }
I20260811 05:17:20.834015 2542 leader_election.cc:302] T 00000000000000000000000000000000 P 17bdf775d0b1467c9401049b0fbae242 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 17bdf775d0b1467c9401049b0fbae242; no voters:
I20260811 05:17:20.834466 2549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:20.832089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:39337 (local address 127.0.0.1:40079)
0811 05:17:20.832403 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:20.832415 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:20.832472 (+ 57us) server_negotiation.cc:407] Connection header received
0811 05:17:20.832650 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:20.832659 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:20.832740 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:20.832922 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:20.833725 (+ 803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:20.833773 (+ 48us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:20.833774 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:20.833784 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:20.833855 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:20.833871 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:20.834008 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:20.834109 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 05:17:20.834164 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":126,"threads_started":1}
I20260811 05:17:20.834486 2542 leader_election.cc:288] T 00000000000000000000000000000000 P 17bdf775d0b1467c9401049b0fbae242 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:17:20.834651 2551 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17bdf775d0b1467c9401049b0fbae242 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:20.834885 2551 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17bdf775d0b1467c9401049b0fbae242 [term 1 LEADER]: Becoming Leader. State: Replica: 17bdf775d0b1467c9401049b0fbae242, State: Running, Role: LEADER
I20260811 05:17:20.835009 2542 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17bdf775d0b1467c9401049b0fbae242 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:20.835047 2551 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17bdf775d0b1467c9401049b0fbae242 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17bdf775d0b1467c9401049b0fbae242" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40079 } }
I20260811 05:17:20.835448 2552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17bdf775d0b1467c9401049b0fbae242 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17bdf775d0b1467c9401049b0fbae242. Latest consensus state: current_term: 1 leader_uuid: "17bdf775d0b1467c9401049b0fbae242" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17bdf775d0b1467c9401049b0fbae242" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40079 } } }
I20260811 05:17:20.835537 2552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17bdf775d0b1467c9401049b0fbae242 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:20.835479 2554 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17bdf775d0b1467c9401049b0fbae242 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "17bdf775d0b1467c9401049b0fbae242" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17bdf775d0b1467c9401049b0fbae242" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40079 } } }
I20260811 05:17:20.835599 2554 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17bdf775d0b1467c9401049b0fbae242 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:20.835662 29246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:17:20.836489 2558 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:17:20.837288 2558 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:17:20.838973 2558 catalog_manager.cc:1347] Generated new cluster ID: 963bed03f62a4e3eb016d986f7b39740
I20260811 05:17:20.839032 2558 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:17:20.839520 2549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:20.838095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:50083 (local address 127.0.0.1:40079)
0811 05:17:20.838134 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:20.838136 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:20.838249 (+ 113us) server_negotiation.cc:407] Connection header received
0811 05:17:20.838479 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:20.838480 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:20.838487 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:20.838536 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:20.839275 (+ 739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:20.839277 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:20.839277 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:20.839282 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:20.839302 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:20.839307 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:20.839382 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:20.839436 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 05:17:20.839452 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 05:17:20.846752 2558 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:17:20.847005 2558 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:17:20.850248 2558 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 17bdf775d0b1467c9401049b0fbae242: Generated new TSK 0
I20260811 05:17:20.850528 2558 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:17:20.854450 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 2412
2026-08-11T05:17:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1502 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425159060450-29246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:17:20 dist-test-slave-f7th krb5kdc[2594](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:17:20 dist-test-slave-f7th krb5kdc[2594](info): set up 2 sockets
Aug 11 05:17:20 dist-test-slave-f7th krb5kdc[2594](info): commencing operation
krb5kdc: starting...
W20260811 05:17:22.079097 29246 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Aug 11 05:17:22 dist-test-slave-f7th krb5kdc[2594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:17:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:17:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 05:17:22.212148 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33869
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33869
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:22.298777 2622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:22.299032 2622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:22.299069 2622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:22.300313 2622 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:22.300367 2622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:22.300385 2622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:22.300401 2622 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:22.300417 2622 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:22.302127 2622 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33869
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:17:22.302424 2622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:22.302651 2622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:22.307636 2632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:22.307633 2636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:22.307669 2630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:22.307999 2622 server_base.cc:1034] running on GCE node
I20260811 05:17:22.308279 2622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:22.308583 2622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:22.309787 2622 hybrid_clock.cc:648] HybridClock initialized: now 1786425442309766 us; error 35 us; skew 500 ppm
Aug 11 05:17:22 dist-test-slave-f7th krb5kdc[2594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425442, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:22.312510 2622 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 05:17:22.313015 2622 webserver.cc:460] Webserver started at http://127.0.0.1:46413/ using document root <none> and password file <none>
I20260811 05:17:22.313230 2622 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:22.313277 2622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:22.313396 2622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:22.314509 2622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "44c13afdaba64ae88483f1d0ee7025d0"
format_stamp: "Formatted at 2026-08-11 05:17:22 on dist-test-slave-f7th"
server_key: "0b9b8d224aff7e35db93d9b7a128a536"
server_key_iv: "613338db8d21b4d089f01abeb052c496"
server_key_version: "encryptionkey@0"
I20260811 05:17:22.314854 2622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "44c13afdaba64ae88483f1d0ee7025d0"
format_stamp: "Formatted at 2026-08-11 05:17:22 on dist-test-slave-f7th"
server_key: "0b9b8d224aff7e35db93d9b7a128a536"
server_key_iv: "613338db8d21b4d089f01abeb052c496"
server_key_version: "encryptionkey@0"
I20260811 05:17:22.316269 2622 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 05:17:22.317216 2651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:22.317453 2622 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 05:17:22.317553 2622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "44c13afdaba64ae88483f1d0ee7025d0"
format_stamp: "Formatted at 2026-08-11 05:17:22 on dist-test-slave-f7th"
server_key: "0b9b8d224aff7e35db93d9b7a128a536"
server_key_iv: "613338db8d21b4d089f01abeb052c496"
server_key_version: "encryptionkey@0"
I20260811 05:17:22.317620 2622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:22.332388 2622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:22.334679 2622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:22.334810 2622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:22.339103 2622 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33869
I20260811 05:17:22.339143 2748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33869 every 8 connection(s)
I20260811 05:17:22.339524 2622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:17:22.340559 2750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:22.343060 2750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44c13afdaba64ae88483f1d0ee7025d0: Bootstrap starting.
I20260811 05:17:22.343704 2750 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44c13afdaba64ae88483f1d0ee7025d0: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:22.343988 2750 log.cc:824] T 00000000000000000000000000000000 P 44c13afdaba64ae88483f1d0ee7025d0: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:22.344894 2750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44c13afdaba64ae88483f1d0ee7025d0: No bootstrap required, opened a new log
I20260811 05:17:22.346609 2750 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44c13afdaba64ae88483f1d0ee7025d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44c13afdaba64ae88483f1d0ee7025d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33869 } }
I20260811 05:17:22.346769 2750 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44c13afdaba64ae88483f1d0ee7025d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:22.346803 2750 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44c13afdaba64ae88483f1d0ee7025d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44c13afdaba64ae88483f1d0ee7025d0, State: Initialized, Role: FOLLOWER
I20260811 05:17:22.346974 2750 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44c13afdaba64ae88483f1d0ee7025d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44c13afdaba64ae88483f1d0ee7025d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33869 } }
I20260811 05:17:22.347065 2750 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44c13afdaba64ae88483f1d0ee7025d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:17:22.347108 2750 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44c13afdaba64ae88483f1d0ee7025d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:17:22.347159 2750 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44c13afdaba64ae88483f1d0ee7025d0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:22.347759 2750 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44c13afdaba64ae88483f1d0ee7025d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44c13afdaba64ae88483f1d0ee7025d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33869 } }
I20260811 05:17:22.347865 2750 leader_election.cc:302] T 00000000000000000000000000000000 P 44c13afdaba64ae88483f1d0ee7025d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44c13afdaba64ae88483f1d0ee7025d0; no voters:
I20260811 05:17:22.348060 2750 leader_election.cc:288] T 00000000000000000000000000000000 P 44c13afdaba64ae88483f1d0ee7025d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:17:22.348109 2753 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44c13afdaba64ae88483f1d0ee7025d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:22.348315 2753 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44c13afdaba64ae88483f1d0ee7025d0 [term 1 LEADER]: Becoming Leader. State: Replica: 44c13afdaba64ae88483f1d0ee7025d0, State: Running, Role: LEADER
I20260811 05:17:22.348330 2750 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44c13afdaba64ae88483f1d0ee7025d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:22.348472 2753 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44c13afdaba64ae88483f1d0ee7025d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44c13afdaba64ae88483f1d0ee7025d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33869 } }
I20260811 05:17:22.348726 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 2622
I20260811 05:17:22.348841 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:17:22.349097 29246 external_mini_cluster.cc:1428] Setting key 21b1a70860d5541ff1b9f39d8b028f1c
I20260811 05:17:22.349112 2756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44c13afdaba64ae88483f1d0ee7025d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44c13afdaba64ae88483f1d0ee7025d0. Latest consensus state: current_term: 1 leader_uuid: "44c13afdaba64ae88483f1d0ee7025d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44c13afdaba64ae88483f1d0ee7025d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33869 } } }
I20260811 05:17:22.349189 2756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44c13afdaba64ae88483f1d0ee7025d0 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:22.349572 2776 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:17:22.350067 2755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44c13afdaba64ae88483f1d0ee7025d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44c13afdaba64ae88483f1d0ee7025d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44c13afdaba64ae88483f1d0ee7025d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33869 } } }
I20260811 05:17:22.350176 2776 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:17:22.350199 2755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44c13afdaba64ae88483f1d0ee7025d0 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:22.351799 2776 catalog_manager.cc:1347] Generated new cluster ID: 0308c922c68e4ca69e0a0d3da024d74c
I20260811 05:17:22.351845 2776 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 05:17:22 dist-test-slave-f7th krb5kdc[2594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 05:17:22.361619 2791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:22.349744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:39329 (local address 127.0.0.1:33869)
0811 05:17:22.350178 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:22.350190 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:22.350223 (+ 33us) server_negotiation.cc:407] Connection header received
0811 05:17:22.350360 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:22.350368 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:22.350433 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:22.350680 (+ 247us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:22.351303 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:22.352699 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:22.353745 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:22.354414 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:22.354535 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:22.357658 (+ 3123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:22.357664 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:22.357665 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:22.357681 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:22.359702 (+ 2021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:22.360241 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:22.360245 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:22.360245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:22.360366 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:22.360608 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:22.360609 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:22.360609 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:22.360959 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:22.361114 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:22.361253 (+ 139us) server_negotiation.cc:299] Negotiation successful
0811 05:17:22.361369 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":75,"threads_started":1}
I20260811 05:17:22.362979 29246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:17:22.366118 2776 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:17:22.366808 2776 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:17:22.372807 2791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:22.364951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:57919 (local address 127.0.0.1:33869)
0811 05:17:22.364989 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:22.364991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:22.365299 (+ 308us) server_negotiation.cc:407] Connection header received
0811 05:17:22.365350 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:22.365351 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:22.365362 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:22.365420 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:22.366138 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:22.366880 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:22.367922 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:22.368238 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:22.368275 (+ 37us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:22.369867 (+ 1592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:22.369869 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:22.369869 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:22.369877 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:22.371444 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:22.371888 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:22.371889 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:22.371890 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:22.371945 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:22.372141 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:22.372142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:22.372143 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:22.372264 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:22.372351 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:22.372593 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 05:17:22.372719 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260811 05:17:22.373410 2776 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44c13afdaba64ae88483f1d0ee7025d0: Generated new TSK 0
I20260811 05:17:22.373584 2776 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:17:22.378882 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 2622
W20260811 05:17:22.403550 2588 connection.cc:535] client connection to 127.0.0.1:33869 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33869): Connection reset by peer (error 104)
2026-08-11T05:17:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1532 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425159060450-29246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:17:22 dist-test-slave-f7th krb5kdc[2813](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:17:22 dist-test-slave-f7th krb5kdc[2813](info): set up 2 sockets
Aug 11 05:17:22 dist-test-slave-f7th krb5kdc[2813](info): commencing operation
krb5kdc: starting...
W20260811 05:17:23.604868 29246 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Aug 11 05:17:23 dist-test-slave-f7th krb5kdc[2813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:17:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:17:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 05:17:23.741900 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36799
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36799
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:23.827991 2833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:23.828212 2833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:23.828236 2833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:23.829413 2833 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:23.829459 2833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:23.829480 2833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:23.829492 2833 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:23.829504 2833 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:23.831012 2833 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36799
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2833
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:17:23.831243 2833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:23.831435 2833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:23.837205 2845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:23.837205 2842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:23.837502 2840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:23.837724 2833 server_base.cc:1034] running on GCE node
I20260811 05:17:23.838008 2833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:23.838317 2833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:23.839496 2833 hybrid_clock.cc:648] HybridClock initialized: now 1786425443839471 us; error 44 us; skew 500 ppm
Aug 11 05:17:23 dist-test-slave-f7th krb5kdc[2813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425443, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:23.842361 2833 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 05:17:23.842854 2833 webserver.cc:460] Webserver started at http://127.0.0.1:36315/ using document root <none> and password file <none>
I20260811 05:17:23.843075 2833 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:23.843120 2833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:23.843248 2833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:23.844359 2833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "65d1864bf7a046c29a08e078ac7741f7"
format_stamp: "Formatted at 2026-08-11 05:17:23 on dist-test-slave-f7th"
server_key: "502e624da81a5782f6afaf4548eee887"
server_key_iv: "5e08df6febfb92a7f0954df8027c3f67"
server_key_version: "encryptionkey@0"
I20260811 05:17:23.844753 2833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "65d1864bf7a046c29a08e078ac7741f7"
format_stamp: "Formatted at 2026-08-11 05:17:23 on dist-test-slave-f7th"
server_key: "502e624da81a5782f6afaf4548eee887"
server_key_iv: "5e08df6febfb92a7f0954df8027c3f67"
server_key_version: "encryptionkey@0"
I20260811 05:17:23.846112 2833 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 05:17:23.846903 2861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:23.847079 2833 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 05:17:23.847173 2833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "65d1864bf7a046c29a08e078ac7741f7"
format_stamp: "Formatted at 2026-08-11 05:17:23 on dist-test-slave-f7th"
server_key: "502e624da81a5782f6afaf4548eee887"
server_key_iv: "5e08df6febfb92a7f0954df8027c3f67"
server_key_version: "encryptionkey@0"
I20260811 05:17:23.847236 2833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:23.861110 2833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:23.863283 2833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:23.863435 2833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:23.867734 2833 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36799
I20260811 05:17:23.867805 2956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36799 every 8 connection(s)
I20260811 05:17:23.868175 2833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:17:23.869206 2960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:23.871683 2960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65d1864bf7a046c29a08e078ac7741f7: Bootstrap starting.
I20260811 05:17:23.872344 2960 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65d1864bf7a046c29a08e078ac7741f7: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:23.872661 2960 log.cc:824] T 00000000000000000000000000000000 P 65d1864bf7a046c29a08e078ac7741f7: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:23.873488 2960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65d1864bf7a046c29a08e078ac7741f7: No bootstrap required, opened a new log
I20260811 05:17:23.875195 2960 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65d1864bf7a046c29a08e078ac7741f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d1864bf7a046c29a08e078ac7741f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36799 } }
I20260811 05:17:23.875355 2960 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65d1864bf7a046c29a08e078ac7741f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:23.875377 2960 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65d1864bf7a046c29a08e078ac7741f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65d1864bf7a046c29a08e078ac7741f7, State: Initialized, Role: FOLLOWER
I20260811 05:17:23.875528 2960 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65d1864bf7a046c29a08e078ac7741f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d1864bf7a046c29a08e078ac7741f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36799 } }
I20260811 05:17:23.875597 2960 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65d1864bf7a046c29a08e078ac7741f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:17:23.875619 2960 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65d1864bf7a046c29a08e078ac7741f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:17:23.875648 2960 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65d1864bf7a046c29a08e078ac7741f7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:23.876238 2960 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65d1864bf7a046c29a08e078ac7741f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d1864bf7a046c29a08e078ac7741f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36799 } }
I20260811 05:17:23.876338 2960 leader_election.cc:302] T 00000000000000000000000000000000 P 65d1864bf7a046c29a08e078ac7741f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65d1864bf7a046c29a08e078ac7741f7; no voters:
I20260811 05:17:23.876528 2960 leader_election.cc:288] T 00000000000000000000000000000000 P 65d1864bf7a046c29a08e078ac7741f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:17:23.876667 2965 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65d1864bf7a046c29a08e078ac7741f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:23.876827 2960 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65d1864bf7a046c29a08e078ac7741f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:23.876843 2965 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65d1864bf7a046c29a08e078ac7741f7 [term 1 LEADER]: Becoming Leader. State: Replica: 65d1864bf7a046c29a08e078ac7741f7, State: Running, Role: LEADER
I20260811 05:17:23.876991 2965 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65d1864bf7a046c29a08e078ac7741f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d1864bf7a046c29a08e078ac7741f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36799 } }
I20260811 05:17:23.877496 2967 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65d1864bf7a046c29a08e078ac7741f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65d1864bf7a046c29a08e078ac7741f7. Latest consensus state: current_term: 1 leader_uuid: "65d1864bf7a046c29a08e078ac7741f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d1864bf7a046c29a08e078ac7741f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36799 } } }
I20260811 05:17:23.877592 2967 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65d1864bf7a046c29a08e078ac7741f7 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:23.877637 2966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65d1864bf7a046c29a08e078ac7741f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "65d1864bf7a046c29a08e078ac7741f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d1864bf7a046c29a08e078ac7741f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36799 } } }
I20260811 05:17:23.877720 2966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65d1864bf7a046c29a08e078ac7741f7 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:23.878134 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 2833
I20260811 05:17:23.878250 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:17:23.878432 29246 external_mini_cluster.cc:1428] Setting key 7a04486782307da8dc85856f62c4c2ad
I20260811 05:17:23.878665 2976 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:17:23.879289 2976 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:17:23.880987 2976 catalog_manager.cc:1347] Generated new cluster ID: 2b20dca9ca4b4d4c846a4e1f3ec3fad0
I20260811 05:17:23.881040 2976 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:17:23.884929 2991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:23.879179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:42709 (local address 127.0.0.1:36799)
0811 05:17:23.879357 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:23.879367 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:23.879408 (+ 41us) server_negotiation.cc:407] Connection header received
0811 05:17:23.879537 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:23.879544 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:23.879605 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:23.879752 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:23.879998 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:23.881410 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:23.882547 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:23.883224 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:23.883366 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:23.884103 (+ 737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:23.884108 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:23.884109 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:23.884352 (+ 243us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:23.884407 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:23.884426 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:23.884526 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:23.884645 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 05:17:23.884682 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":83,"threads_started":1}
I20260811 05:17:23.885919 29246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:17:23.890775 2976 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:17:23.891422 2976 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:17:23.891893 2991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:23.887764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:51511 (local address 127.0.0.1:36799)
0811 05:17:23.887804 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:23.887806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:23.887939 (+ 133us) server_negotiation.cc:407] Connection header received
0811 05:17:23.888003 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:23.888004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:23.888029 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:23.888102 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:23.888263 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:23.889116 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:23.890397 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:23.890675 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:23.890756 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:23.891628 (+ 872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:23.891630 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:23.891630 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:23.891636 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:23.891656 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:23.891661 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:23.891726 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:23.891797 (+ 71us) server_negotiation.cc:299] Negotiation successful
0811 05:17:23.891811 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 05:17:23.894642 2976 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 65d1864bf7a046c29a08e078ac7741f7: Generated new TSK 0
I20260811 05:17:23.894862 2976 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:17:23.902820 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 2833
W20260811 05:17:23.926899 2808 connection.cc:535] client connection to 127.0.0.1:36799 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36799): Connection reset by peer (error 104)
2026-08-11T05:17:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1523 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425159060450-29246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:17:23 dist-test-slave-f7th krb5kdc[3002](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:17:23 dist-test-slave-f7th krb5kdc[3002](info): set up 2 sockets
Aug 11 05:17:23 dist-test-slave-f7th krb5kdc[3002](info): commencing operation
krb5kdc: starting...
W20260811 05:17:25.133724 29246 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Aug 11 05:17:25 dist-test-slave-f7th krb5kdc[3002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:17:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:17:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 05:17:25.269194 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37863
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37863
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:25.355199 3023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:25.355410 3023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:25.355441 3023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:25.356642 3023 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:25.356689 3023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:25.356702 3023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:25.356714 3023 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:25.356724 3023 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:25.358281 3023 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37863
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425159060450-29246-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3023
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:17:25.358534 3023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:25.358765 3023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:25.363754 3034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:25.363760 3035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:25.363888 3023 server_base.cc:1034] running on GCE node
W20260811 05:17:25.363831 3039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:25.364336 3023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:25.364715 3023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:25.365885 3023 hybrid_clock.cc:648] HybridClock initialized: now 1786425445365865 us; error 30 us; skew 500 ppm
Aug 11 05:17:25 dist-test-slave-f7th krb5kdc[3002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425445, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:25.373399 3023 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 05:17:25.373939 3023 webserver.cc:460] Webserver started at http://127.0.0.1:39279/ using document root <none> and password file <none>
I20260811 05:17:25.374189 3023 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:25.374243 3023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:25.374377 3023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:25.375560 3023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "d5d76b765c0d41818fd1952360552ea6"
format_stamp: "Formatted at 2026-08-11 05:17:25 on dist-test-slave-f7th"
server_key: "03ac20c7bd506cf4d3ebd9282830382a"
server_key_iv: "da78806b7f6a4163ed808a47d582fd5e"
server_key_version: "encryptionkey@0"
I20260811 05:17:25.375905 3023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "d5d76b765c0d41818fd1952360552ea6"
format_stamp: "Formatted at 2026-08-11 05:17:25 on dist-test-slave-f7th"
server_key: "03ac20c7bd506cf4d3ebd9282830382a"
server_key_iv: "da78806b7f6a4163ed808a47d582fd5e"
server_key_version: "encryptionkey@0"
I20260811 05:17:25.377362 3023 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 05:17:25.378185 3049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:25.378360 3023 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:17:25.378468 3023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "d5d76b765c0d41818fd1952360552ea6"
format_stamp: "Formatted at 2026-08-11 05:17:25 on dist-test-slave-f7th"
server_key: "03ac20c7bd506cf4d3ebd9282830382a"
server_key_iv: "da78806b7f6a4163ed808a47d582fd5e"
server_key_version: "encryptionkey@0"
I20260811 05:17:25.378561 3023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:25.382134 3023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:25.384199 3023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:25.384341 3023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:25.388465 3023 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37863
I20260811 05:17:25.388504 3151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37863 every 8 connection(s)
I20260811 05:17:25.388895 3023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:17:25.389881 3157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:25.392383 3157 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5d76b765c0d41818fd1952360552ea6: Bootstrap starting.
I20260811 05:17:25.393075 3157 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5d76b765c0d41818fd1952360552ea6: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:25.393368 3157 log.cc:824] T 00000000000000000000000000000000 P d5d76b765c0d41818fd1952360552ea6: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:25.394212 3157 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5d76b765c0d41818fd1952360552ea6: No bootstrap required, opened a new log
I20260811 05:17:25.395763 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 3023
I20260811 05:17:25.395876 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:17:25.396116 29246 external_mini_cluster.cc:1428] Setting key 29860aed977a46def9c1f302021a1200
I20260811 05:17:25.396000 3157 raft_consensus.cc:348] T 00000000000000000000000000000000 P d5d76b765c0d41818fd1952360552ea6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5d76b765c0d41818fd1952360552ea6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37863 } }
I20260811 05:17:25.396193 3157 raft_consensus.cc:374] T 00000000000000000000000000000000 P d5d76b765c0d41818fd1952360552ea6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:25.396236 3157 raft_consensus.cc:729] T 00000000000000000000000000000000 P d5d76b765c0d41818fd1952360552ea6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5d76b765c0d41818fd1952360552ea6, State: Initialized, Role: FOLLOWER
I20260811 05:17:25.396436 3157 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5d76b765c0d41818fd1952360552ea6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5d76b765c0d41818fd1952360552ea6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37863 } }
I20260811 05:17:25.396533 3157 raft_consensus.cc:388] T 00000000000000000000000000000000 P d5d76b765c0d41818fd1952360552ea6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:17:25.396567 3157 raft_consensus.cc:482] T 00000000000000000000000000000000 P d5d76b765c0d41818fd1952360552ea6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:17:25.396608 3157 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d5d76b765c0d41818fd1952360552ea6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:25.397344 3157 raft_consensus.cc:504] T 00000000000000000000000000000000 P d5d76b765c0d41818fd1952360552ea6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5d76b765c0d41818fd1952360552ea6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37863 } }
I20260811 05:17:25.397460 3157 leader_election.cc:302] T 00000000000000000000000000000000 P d5d76b765c0d41818fd1952360552ea6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d5d76b765c0d41818fd1952360552ea6; no voters:
I20260811 05:17:25.397651 3157 leader_election.cc:288] T 00000000000000000000000000000000 P d5d76b765c0d41818fd1952360552ea6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:17:25.397696 3165 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d5d76b765c0d41818fd1952360552ea6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:25.397851 3165 raft_consensus.cc:686] T 00000000000000000000000000000000 P d5d76b765c0d41818fd1952360552ea6 [term 1 LEADER]: Becoming Leader. State: Replica: d5d76b765c0d41818fd1952360552ea6, State: Running, Role: LEADER
I20260811 05:17:25.397998 3165 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5d76b765c0d41818fd1952360552ea6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5d76b765c0d41818fd1952360552ea6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37863 } }
I20260811 05:17:25.398046 3157 sys_catalog.cc:564] T 00000000000000000000000000000000 P d5d76b765c0d41818fd1952360552ea6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:25.398833 3164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:25.396667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:35807 (local address 127.0.0.1:37863)
0811 05:17:25.396875 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:25.396887 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:25.396923 (+ 36us) server_negotiation.cc:407] Connection header received
0811 05:17:25.397271 (+ 348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:25.397278 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:25.397306 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:25.397454 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:25.398198 (+ 744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:25.398203 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:25.398203 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:25.398208 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:25.398256 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:25.398271 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:25.398344 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:25.398572 (+ 228us) server_negotiation.cc:299] Negotiation successful
0811 05:17:25.398625 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":92,"threads_started":1}
I20260811 05:17:25.399439 3167 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5d76b765c0d41818fd1952360552ea6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5d76b765c0d41818fd1952360552ea6. Latest consensus state: current_term: 1 leader_uuid: "d5d76b765c0d41818fd1952360552ea6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5d76b765c0d41818fd1952360552ea6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37863 } } }
I20260811 05:17:25.399703 3167 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5d76b765c0d41818fd1952360552ea6 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:25.399634 3166 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5d76b765c0d41818fd1952360552ea6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d5d76b765c0d41818fd1952360552ea6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5d76b765c0d41818fd1952360552ea6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37863 } } }
I20260811 05:17:25.399868 29246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:17:25.399927 3166 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5d76b765c0d41818fd1952360552ea6 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:25.400048 3194 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:17:25.400761 3194 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:17:25.403036 3164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:25.401573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:55119 (local address 127.0.0.1:37863)
0811 05:17:25.401607 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:25.401609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:25.402106 (+ 497us) server_negotiation.cc:407] Connection header received
0811 05:17:25.402163 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:25.402164 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:25.402170 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:25.402240 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:25.402832 (+ 592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:25.402833 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:25.402834 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:25.402838 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:25.402857 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:25.402862 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:25.402920 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:25.402958 (+ 38us) server_negotiation.cc:299] Negotiation successful
0811 05:17:25.402972 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260811 05:17:25.405177 3194 catalog_manager.cc:1347] Generated new cluster ID: 57c88aa5ccb24e24b16ef857123960eb
I20260811 05:17:25.405238 3194 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:17:25.428596 3194 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:17:25.428840 3194 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:17:25.433234 3194 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d5d76b765c0d41818fd1952360552ea6: Generated new TSK 0
I20260811 05:17:25.433480 3194 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:17:25.436146 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 3023
2026-08-11T05:17:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1527 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425159060450-29246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:17:25 dist-test-slave-f7th krb5kdc[3215](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:17:25 dist-test-slave-f7th krb5kdc[3215](info): set up 2 sockets
Aug 11 05:17:25 dist-test-slave-f7th krb5kdc[3215](info): commencing operation
krb5kdc: starting...
W20260811 05:17:26.653522 29246 mini_kdc.cc:121] Time spent starting KDC: real 1.185s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Aug 11 05:17:26 dist-test-slave-f7th krb5kdc[3215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:17:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:17:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
I20260811 05:17:26.789160 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:45021
--webserver_interface=127.28.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:45635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:45021
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:26.882229 3238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:26.882486 3238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:26.882513 3238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:26.884030 3238 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:26.884127 3238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:26.884142 3238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:26.884155 3238 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:26.884166 3238 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:26.886013 3238 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:45635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:45021
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:45021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:17:26.886292 3238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:26.886502 3238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:26.891798 3249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:26.891812 3244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:26.891953 3245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:26.891963 3238 server_base.cc:1034] running on GCE node
I20260811 05:17:26.892374 3238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:26.892716 3238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:26.893882 3238 hybrid_clock.cc:648] HybridClock initialized: now 1786425446893846 us; error 60 us; skew 500 ppm
Aug 11 05:17:26 dist-test-slave-f7th krb5kdc[3215](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425446, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:26.896788 3238 init.cc:377] Logged in from keytab as kudu/127.28.143.190@KRBTEST.COM (short username kudu)
I20260811 05:17:26.897362 3238 webserver.cc:460] Webserver started at http://127.28.143.190:35319/ using document root <none> and password file <none>
I20260811 05:17:26.897586 3238 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:26.897634 3238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:26.897764 3238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:26.898931 3238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "469c65ffabf0489bb6c43e368104661d"
format_stamp: "Formatted at 2026-08-11 05:17:26 on dist-test-slave-f7th"
server_key: "f2e1778bdb0255bfe56866199e00482a"
server_key_iv: "cef7dcf52b0926ccc7f3843c575c9b65"
server_key_version: "encryptionkey@0"
I20260811 05:17:26.899338 3238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "469c65ffabf0489bb6c43e368104661d"
format_stamp: "Formatted at 2026-08-11 05:17:26 on dist-test-slave-f7th"
server_key: "f2e1778bdb0255bfe56866199e00482a"
server_key_iv: "cef7dcf52b0926ccc7f3843c575c9b65"
server_key_version: "encryptionkey@0"
I20260811 05:17:26.900813 3238 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 05:17:26.901810 3264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:26.902082 3238 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:17:26.902190 3238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "469c65ffabf0489bb6c43e368104661d"
format_stamp: "Formatted at 2026-08-11 05:17:26 on dist-test-slave-f7th"
server_key: "f2e1778bdb0255bfe56866199e00482a"
server_key_iv: "cef7dcf52b0926ccc7f3843c575c9b65"
server_key_version: "encryptionkey@0"
I20260811 05:17:26.902269 3238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:26.929201 3238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:26.931775 3238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:26.932003 3238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:26.936219 3238 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:45021
I20260811 05:17:26.936277 3369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:45021 every 8 connection(s)
I20260811 05:17:26.936650 3238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:17:26.937598 3370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:26.940092 3370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 469c65ffabf0489bb6c43e368104661d: Bootstrap starting.
I20260811 05:17:26.940759 3370 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 469c65ffabf0489bb6c43e368104661d: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:26.941047 3370 log.cc:824] T 00000000000000000000000000000000 P 469c65ffabf0489bb6c43e368104661d: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:26.941943 3370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 469c65ffabf0489bb6c43e368104661d: No bootstrap required, opened a new log
I20260811 05:17:26.943637 3370 raft_consensus.cc:348] T 00000000000000000000000000000000 P 469c65ffabf0489bb6c43e368104661d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "469c65ffabf0489bb6c43e368104661d" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 45021 } }
I20260811 05:17:26.943799 3370 raft_consensus.cc:374] T 00000000000000000000000000000000 P 469c65ffabf0489bb6c43e368104661d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:26.943821 3370 raft_consensus.cc:729] T 00000000000000000000000000000000 P 469c65ffabf0489bb6c43e368104661d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 469c65ffabf0489bb6c43e368104661d, State: Initialized, Role: FOLLOWER
I20260811 05:17:26.943972 3370 consensus_queue.cc:260] T 00000000000000000000000000000000 P 469c65ffabf0489bb6c43e368104661d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "469c65ffabf0489bb6c43e368104661d" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 45021 } }
I20260811 05:17:26.944043 3370 raft_consensus.cc:388] T 00000000000000000000000000000000 P 469c65ffabf0489bb6c43e368104661d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:17:26.944067 3370 raft_consensus.cc:482] T 00000000000000000000000000000000 P 469c65ffabf0489bb6c43e368104661d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:17:26.944118 3370 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 469c65ffabf0489bb6c43e368104661d [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:26.945093 3370 raft_consensus.cc:504] T 00000000000000000000000000000000 P 469c65ffabf0489bb6c43e368104661d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "469c65ffabf0489bb6c43e368104661d" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 45021 } }
I20260811 05:17:26.945202 3370 leader_election.cc:302] T 00000000000000000000000000000000 P 469c65ffabf0489bb6c43e368104661d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 469c65ffabf0489bb6c43e368104661d; no voters:
I20260811 05:17:26.945382 3370 leader_election.cc:288] T 00000000000000000000000000000000 P 469c65ffabf0489bb6c43e368104661d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:17:26.945431 3378 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 469c65ffabf0489bb6c43e368104661d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:26.945608 3378 raft_consensus.cc:686] T 00000000000000000000000000000000 P 469c65ffabf0489bb6c43e368104661d [term 1 LEADER]: Becoming Leader. State: Replica: 469c65ffabf0489bb6c43e368104661d, State: Running, Role: LEADER
I20260811 05:17:26.945676 3370 sys_catalog.cc:564] T 00000000000000000000000000000000 P 469c65ffabf0489bb6c43e368104661d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:26.945717 3378 consensus_queue.cc:237] T 00000000000000000000000000000000 P 469c65ffabf0489bb6c43e368104661d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "469c65ffabf0489bb6c43e368104661d" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 45021 } }
I20260811 05:17:26.946143 3381 sys_catalog.cc:455] T 00000000000000000000000000000000 P 469c65ffabf0489bb6c43e368104661d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 469c65ffabf0489bb6c43e368104661d. Latest consensus state: current_term: 1 leader_uuid: "469c65ffabf0489bb6c43e368104661d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "469c65ffabf0489bb6c43e368104661d" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 45021 } } }
I20260811 05:17:26.946233 3381 sys_catalog.cc:458] T 00000000000000000000000000000000 P 469c65ffabf0489bb6c43e368104661d [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:26.946126 3380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 469c65ffabf0489bb6c43e368104661d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "469c65ffabf0489bb6c43e368104661d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "469c65ffabf0489bb6c43e368104661d" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 45021 } } }
I20260811 05:17:26.946319 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 3238
I20260811 05:17:26.946362 3380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 469c65ffabf0489bb6c43e368104661d [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:26.946442 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:17:26.946506 3384 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:17:26.946614 29246 external_mini_cluster.cc:1428] Setting key d8cb5da1f1287f95cf424c33b42a6200
I20260811 05:17:26.947086 3384 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:17:26.949122 3384 catalog_manager.cc:1347] Generated new cluster ID: b70e81e7f50044f3b58794d06ea9402d
I20260811 05:17:26.949182 3384 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 05:17:26 dist-test-slave-f7th krb5kdc[3215](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:17:26.959532 3400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:26.947303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60625 (local address 127.28.143.190:45021)
0811 05:17:26.947463 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:26.947473 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:26.947505 (+ 32us) server_negotiation.cc:407] Connection header received
0811 05:17:26.948109 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:26.948120 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:26.948191 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:26.948344 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:26.949166 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:26.950553 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:26.951696 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:26.952374 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:26.952484 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:26.955488 (+ 3004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:26.955495 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:26.955496 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:26.955512 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:26.957734 (+ 2222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:26.958214 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:26.958218 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:26.958218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:26.958338 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:26.958548 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:26.958549 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:26.958549 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:26.958894 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:26.959079 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:26.959162 (+ 83us) server_negotiation.cc:299] Negotiation successful
0811 05:17:26.959276 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":66,"threads_started":1}
I20260811 05:17:26.960680 29246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:17:26.968916 3384 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:17:26.969762 3384 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:17:26.975165 3400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:26.968547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50887 (local address 127.28.143.190:45021)
0811 05:17:26.968609 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:26.968631 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:26.968966 (+ 335us) server_negotiation.cc:407] Connection header received
0811 05:17:26.969002 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:26.969003 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:26.969013 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:26.969062 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:26.969669 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:26.970224 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:26.970927 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:26.971190 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:26.971249 (+ 59us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:26.972560 (+ 1311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:26.972561 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:26.972561 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:26.972570 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:26.974026 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:26.974395 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:26.974396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:26.974397 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:26.974458 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:26.974662 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:26.974663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:26.974664 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:26.974779 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:26.974896 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:26.975008 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 05:17:26.975073 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 05:17:26.977981 3384 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 469c65ffabf0489bb6c43e368104661d: Generated new TSK 0
I20260811 05:17:26.978426 3384 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:17:26.980980 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 3238
W20260811 05:17:27.006199 3212 connection.cc:535] client connection to 127.28.143.190:45021 recv error: Network error: failed to read from TLS socket (remote: 127.28.143.190:45021): Connection reset by peer (error 104)
2026-08-11T05:17:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1552 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425159060450-29246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:17:27 dist-test-slave-f7th krb5kdc[3422](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:17:27 dist-test-slave-f7th krb5kdc[3422](info): set up 2 sockets
Aug 11 05:17:27 dist-test-slave-f7th krb5kdc[3422](info): commencing operation
krb5kdc: starting...
W20260811 05:17:28.216603 29246 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.003s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Aug 11 05:17:28 dist-test-slave-f7th krb5kdc[3422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:17:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:17:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
I20260811 05:17:28.352905 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:42453
--webserver_interface=127.28.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:33497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:42453
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:28.439532 3446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:28.439781 3446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:28.439806 3446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:28.441013 3446 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:28.441059 3446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:28.441072 3446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:28.441092 3446 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:28.441103 3446 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:28.442670 3446 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:33497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:42453
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:42453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3446
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:17:28.442900 3446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:28.443105 3446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:28.448352 3460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:28.448339 3458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:28.448408 3446 server_base.cc:1034] running on GCE node
W20260811 05:17:28.448352 3457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:28.448889 3446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:28.449231 3446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:28.450431 3446 hybrid_clock.cc:648] HybridClock initialized: now 1786425448450418 us; error 32 us; skew 500 ppm
Aug 11 05:17:28 dist-test-slave-f7th krb5kdc[3422](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425448, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:28.453189 3446 init.cc:377] Logged in from keytab as kudu/127.28.143.190@KRBTEST.COM (short username kudu)
I20260811 05:17:28.453729 3446 webserver.cc:460] Webserver started at http://127.28.143.190:37687/ using document root <none> and password file <none>
I20260811 05:17:28.453969 3446 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:28.454026 3446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:28.454161 3446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:28.455253 3446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "468081f2099b4c11acaf057cfcc88d92"
format_stamp: "Formatted at 2026-08-11 05:17:28 on dist-test-slave-f7th"
server_key: "5223978eb508a41660c6bdac6659978f"
server_key_iv: "70f7e0cbe41a19d63b25f3ce64d2508d"
server_key_version: "encryptionkey@0"
I20260811 05:17:28.455605 3446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "468081f2099b4c11acaf057cfcc88d92"
format_stamp: "Formatted at 2026-08-11 05:17:28 on dist-test-slave-f7th"
server_key: "5223978eb508a41660c6bdac6659978f"
server_key_iv: "70f7e0cbe41a19d63b25f3ce64d2508d"
server_key_version: "encryptionkey@0"
I20260811 05:17:28.456975 3446 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 05:17:28.457708 3472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:28.457877 3446 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 05:17:28.457971 3446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "468081f2099b4c11acaf057cfcc88d92"
format_stamp: "Formatted at 2026-08-11 05:17:28 on dist-test-slave-f7th"
server_key: "5223978eb508a41660c6bdac6659978f"
server_key_iv: "70f7e0cbe41a19d63b25f3ce64d2508d"
server_key_version: "encryptionkey@0"
I20260811 05:17:28.458034 3446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:28.472808 3446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:28.475005 3446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:28.475168 3446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:28.479372 3446 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:42453
I20260811 05:17:28.479420 3571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:42453 every 8 connection(s)
I20260811 05:17:28.479799 3446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:17:28.480793 3572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:28.483227 3572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 468081f2099b4c11acaf057cfcc88d92: Bootstrap starting.
I20260811 05:17:28.483820 3572 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 468081f2099b4c11acaf057cfcc88d92: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:28.484066 3572 log.cc:824] T 00000000000000000000000000000000 P 468081f2099b4c11acaf057cfcc88d92: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:28.484894 3572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 468081f2099b4c11acaf057cfcc88d92: No bootstrap required, opened a new log
I20260811 05:17:28.486577 3572 raft_consensus.cc:348] T 00000000000000000000000000000000 P 468081f2099b4c11acaf057cfcc88d92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "468081f2099b4c11acaf057cfcc88d92" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 42453 } }
I20260811 05:17:28.486740 3572 raft_consensus.cc:374] T 00000000000000000000000000000000 P 468081f2099b4c11acaf057cfcc88d92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:28.486763 3572 raft_consensus.cc:729] T 00000000000000000000000000000000 P 468081f2099b4c11acaf057cfcc88d92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 468081f2099b4c11acaf057cfcc88d92, State: Initialized, Role: FOLLOWER
I20260811 05:17:28.486892 3572 consensus_queue.cc:260] T 00000000000000000000000000000000 P 468081f2099b4c11acaf057cfcc88d92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "468081f2099b4c11acaf057cfcc88d92" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 42453 } }
I20260811 05:17:28.486951 3572 raft_consensus.cc:388] T 00000000000000000000000000000000 P 468081f2099b4c11acaf057cfcc88d92 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:17:28.486972 3572 raft_consensus.cc:482] T 00000000000000000000000000000000 P 468081f2099b4c11acaf057cfcc88d92 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:17:28.487016 3572 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 468081f2099b4c11acaf057cfcc88d92 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:28.487926 3572 raft_consensus.cc:504] T 00000000000000000000000000000000 P 468081f2099b4c11acaf057cfcc88d92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "468081f2099b4c11acaf057cfcc88d92" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 42453 } }
I20260811 05:17:28.488042 3572 leader_election.cc:302] T 00000000000000000000000000000000 P 468081f2099b4c11acaf057cfcc88d92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 468081f2099b4c11acaf057cfcc88d92; no voters:
I20260811 05:17:28.488221 3572 leader_election.cc:288] T 00000000000000000000000000000000 P 468081f2099b4c11acaf057cfcc88d92 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:17:28.488269 3580 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 468081f2099b4c11acaf057cfcc88d92 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:28.488461 3580 raft_consensus.cc:686] T 00000000000000000000000000000000 P 468081f2099b4c11acaf057cfcc88d92 [term 1 LEADER]: Becoming Leader. State: Replica: 468081f2099b4c11acaf057cfcc88d92, State: Running, Role: LEADER
I20260811 05:17:28.488505 3572 sys_catalog.cc:564] T 00000000000000000000000000000000 P 468081f2099b4c11acaf057cfcc88d92 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:28.488569 3580 consensus_queue.cc:237] T 00000000000000000000000000000000 P 468081f2099b4c11acaf057cfcc88d92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "468081f2099b4c11acaf057cfcc88d92" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 42453 } }
I20260811 05:17:28.489018 3583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 468081f2099b4c11acaf057cfcc88d92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 468081f2099b4c11acaf057cfcc88d92. Latest consensus state: current_term: 1 leader_uuid: "468081f2099b4c11acaf057cfcc88d92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "468081f2099b4c11acaf057cfcc88d92" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 42453 } } }
I20260811 05:17:28.489111 3583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 468081f2099b4c11acaf057cfcc88d92 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:28.489305 3581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 468081f2099b4c11acaf057cfcc88d92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "468081f2099b4c11acaf057cfcc88d92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "468081f2099b4c11acaf057cfcc88d92" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 42453 } } }
I20260811 05:17:28.489420 3581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 468081f2099b4c11acaf057cfcc88d92 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:28.489754 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 3446
I20260811 05:17:28.489854 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:17:28.490061 29246 external_mini_cluster.cc:1428] Setting key 7809bda49f228e3c4aec97864c73bda5
I20260811 05:17:28.490442 3596 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:17:28.491057 3596 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:17:28.492774 3596 catalog_manager.cc:1347] Generated new cluster ID: 7b0b7bd1453644cea79498b8605b8238
I20260811 05:17:28.492825 3596 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:17:28.495597 3613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:28.490789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49297 (local address 127.28.143.190:42453)
0811 05:17:28.491033 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:28.491045 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:28.491080 (+ 35us) server_negotiation.cc:407] Connection header received
0811 05:17:28.491223 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:28.491230 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:28.491289 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:28.491426 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:28.491654 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:28.492764 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:28.493626 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:28.494079 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:28.494178 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:28.494893 (+ 715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:28.494898 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:28.494898 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:28.495086 (+ 188us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:28.495118 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:28.495131 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:28.495204 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:28.495300 (+ 96us) server_negotiation.cc:299] Negotiation successful
0811 05:17:28.495417 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":134,"threads_started":1}
I20260811 05:17:28.496492 29246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:17:28.507689 3613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:28.501047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33345 (local address 127.28.143.190:42453)
0811 05:17:28.501104 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:28.501106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:28.504860 (+ 3754us) server_negotiation.cc:407] Connection header received
0811 05:17:28.504931 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:28.504933 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:28.504946 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:28.504991 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:28.505134 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:28.505699 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:28.506455 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:28.506673 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:28.506731 (+ 58us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:28.507432 (+ 701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:28.507434 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:28.507434 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:28.507439 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:28.507457 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:28.507462 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:28.507508 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:28.507580 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 05:17:28.507593 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 05:17:28.520390 3596 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:17:28.521212 3596 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:17:28.532964 3596 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 468081f2099b4c11acaf057cfcc88d92: Generated new TSK 0
I20260811 05:17:28.533211 3596 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:17:28.537724 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 3446
W20260811 05:17:28.579321 3420 connection.cc:535] client connection to 127.28.143.190:42453 recv error: Network error: failed to read from TLS socket (remote: 127.28.143.190:42453): Connection reset by peer (error 104)
2026-08-11T05:17:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1572 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425159060450-29246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:17:28 dist-test-slave-f7th krb5kdc[3626](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:17:28 dist-test-slave-f7th krb5kdc[3626](info): set up 2 sockets
Aug 11 05:17:28 dist-test-slave-f7th krb5kdc[3626](info): commencing operation
krb5kdc: starting...
W20260811 05:17:29.780532 29246 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Aug 11 05:17:29 dist-test-slave-f7th krb5kdc[3626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425449, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:17:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:17:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
I20260811 05:17:29.921663 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:39417
--webserver_interface=127.28.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:46521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:39417
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:30.010799 3654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:30.011061 3654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:30.011088 3654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:30.012437 3654 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:30.012503 3654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:30.012517 3654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:30.012537 3654 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:30.012547 3654 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:30.014375 3654 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:46521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:39417
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:39417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:17:30.014714 3654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:30.014966 3654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:30.020434 3663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:30.020558 3654 server_base.cc:1034] running on GCE node
W20260811 05:17:30.020452 3670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:30.020486 3662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:30.021013 3654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:30.021343 3654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:30.022536 3654 hybrid_clock.cc:648] HybridClock initialized: now 1786425450022518 us; error 35 us; skew 500 ppm
Aug 11 05:17:30 dist-test-slave-f7th krb5kdc[3626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425450, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:30.025800 3654 init.cc:377] Logged in from keytab as kudu/127.28.143.190@KRBTEST.COM (short username kudu)
I20260811 05:17:30.026408 3654 webserver.cc:460] Webserver started at http://127.28.143.190:40171/ using document root <none> and password file <none>
I20260811 05:17:30.026669 3654 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:30.026726 3654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:30.026886 3654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:30.028107 3654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "3063d280c692486db8d78ac2050874dc"
format_stamp: "Formatted at 2026-08-11 05:17:30 on dist-test-slave-f7th"
server_key: "5f9fca3e2e6bd5a70b1628236f5953c0"
server_key_iv: "01e15cb261f1b62545ed7780faeb6055"
server_key_version: "encryptionkey@0"
I20260811 05:17:30.028469 3654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "3063d280c692486db8d78ac2050874dc"
format_stamp: "Formatted at 2026-08-11 05:17:30 on dist-test-slave-f7th"
server_key: "5f9fca3e2e6bd5a70b1628236f5953c0"
server_key_iv: "01e15cb261f1b62545ed7780faeb6055"
server_key_version: "encryptionkey@0"
I20260811 05:17:30.030062 3654 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 05:17:30.031013 3684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:30.031293 3654 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 05:17:30.031409 3654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "3063d280c692486db8d78ac2050874dc"
format_stamp: "Formatted at 2026-08-11 05:17:30 on dist-test-slave-f7th"
server_key: "5f9fca3e2e6bd5a70b1628236f5953c0"
server_key_iv: "01e15cb261f1b62545ed7780faeb6055"
server_key_version: "encryptionkey@0"
I20260811 05:17:30.031483 3654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:30.035312 3654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:30.037540 3654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:30.037693 3654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:30.041878 3654 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:39417
I20260811 05:17:30.041939 3779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:39417 every 8 connection(s)
I20260811 05:17:30.042279 3654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:17:30.043372 3780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:30.045921 3780 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3063d280c692486db8d78ac2050874dc: Bootstrap starting.
I20260811 05:17:30.046545 3780 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3063d280c692486db8d78ac2050874dc: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:30.046852 3780 log.cc:824] T 00000000000000000000000000000000 P 3063d280c692486db8d78ac2050874dc: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:30.047605 3780 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3063d280c692486db8d78ac2050874dc: No bootstrap required, opened a new log
I20260811 05:17:30.047997 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 3654
I20260811 05:17:30.048094 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:17:30.048264 29246 external_mini_cluster.cc:1428] Setting key 75b5e0140441ff8d213c0209457379ea
I20260811 05:17:30.050120 3780 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3063d280c692486db8d78ac2050874dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3063d280c692486db8d78ac2050874dc" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39417 } }
I20260811 05:17:30.050315 3780 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3063d280c692486db8d78ac2050874dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:30.050359 3780 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3063d280c692486db8d78ac2050874dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3063d280c692486db8d78ac2050874dc, State: Initialized, Role: FOLLOWER
I20260811 05:17:30.050554 3780 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3063d280c692486db8d78ac2050874dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3063d280c692486db8d78ac2050874dc" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39417 } }
I20260811 05:17:30.050652 3780 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3063d280c692486db8d78ac2050874dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:17:30.050722 3780 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3063d280c692486db8d78ac2050874dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:17:30.050776 3780 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3063d280c692486db8d78ac2050874dc [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:30.051407 3784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:30.048826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60953 (local address 127.28.143.190:39417)
0811 05:17:30.049010 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:30.049018 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:30.049053 (+ 35us) server_negotiation.cc:407] Connection header received
0811 05:17:30.050033 (+ 980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:30.050043 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:30.050074 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:30.050206 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:30.050975 (+ 769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:30.050979 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:30.050979 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:30.050983 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:30.051032 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:30.051042 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:30.051099 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:30.051173 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 05:17:30.051209 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":66,"threads_started":1}
I20260811 05:17:30.051502 3780 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3063d280c692486db8d78ac2050874dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3063d280c692486db8d78ac2050874dc" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39417 } }
I20260811 05:17:30.051620 3780 leader_election.cc:302] T 00000000000000000000000000000000 P 3063d280c692486db8d78ac2050874dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3063d280c692486db8d78ac2050874dc; no voters:
I20260811 05:17:30.051788 3780 leader_election.cc:288] T 00000000000000000000000000000000 P 3063d280c692486db8d78ac2050874dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:17:30.051837 3785 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3063d280c692486db8d78ac2050874dc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:30.051972 3785 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3063d280c692486db8d78ac2050874dc [term 1 LEADER]: Becoming Leader. State: Replica: 3063d280c692486db8d78ac2050874dc, State: Running, Role: LEADER
I20260811 05:17:30.052090 3780 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3063d280c692486db8d78ac2050874dc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:30.052124 3785 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3063d280c692486db8d78ac2050874dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3063d280c692486db8d78ac2050874dc" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39417 } }
I20260811 05:17:30.052446 29246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:17:30.052513 3785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3063d280c692486db8d78ac2050874dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3063d280c692486db8d78ac2050874dc. Latest consensus state: current_term: 1 leader_uuid: "3063d280c692486db8d78ac2050874dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3063d280c692486db8d78ac2050874dc" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39417 } } }
I20260811 05:17:30.052599 3785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3063d280c692486db8d78ac2050874dc [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:30.052531 3788 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3063d280c692486db8d78ac2050874dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3063d280c692486db8d78ac2050874dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3063d280c692486db8d78ac2050874dc" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 39417 } } }
I20260811 05:17:30.052685 3788 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3063d280c692486db8d78ac2050874dc [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:30.052822 3796 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:17:30.053385 3796 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:17:30.055066 3825 mvcc.cc:205] Tried to move back new op lower bound from 7317198643423698944 to 7317198643413991424. Current Snapshot: MvccSnapshot[applied={T|T < 7317198643423698944}]
I20260811 05:17:30.055862 3796 catalog_manager.cc:1347] Generated new cluster ID: c45fb8b88819432b9b63951c5b6523b1
I20260811 05:17:30.055938 3796 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:17:30.055948 3784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:30.054235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58729 (local address 127.28.143.190:39417)
0811 05:17:30.054305 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:30.054307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:30.054668 (+ 361us) server_negotiation.cc:407] Connection header received
0811 05:17:30.054774 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:30.054775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:30.054783 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:30.054812 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:30.055637 (+ 825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:30.055639 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:30.055639 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:30.055645 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:30.055661 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:30.055665 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:30.055715 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:30.055806 (+ 91us) server_negotiation.cc:299] Negotiation successful
0811 05:17:30.055821 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 05:17:30.064227 3796 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:17:30.064416 3796 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:17:30.066618 3796 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3063d280c692486db8d78ac2050874dc: Generated new TSK 0
I20260811 05:17:30.066797 3796 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:17:30.071805 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 3654
2026-08-11T05:17:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1512 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425159060450-29246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:17:30 dist-test-slave-f7th krb5kdc[3832](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:17:30 dist-test-slave-f7th krb5kdc[3832](info): set up 2 sockets
Aug 11 05:17:30 dist-test-slave-f7th krb5kdc[3832](info): commencing operation
krb5kdc: starting...
W20260811 05:17:31.300160 29246 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Aug 11 05:17:31 dist-test-slave-f7th krb5kdc[3832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425451, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:17:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:17:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
I20260811 05:17:31.436254 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:46661
--webserver_interface=127.28.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:37945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:46661
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:31.523433 3859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:31.523684 3859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:31.523720 3859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:31.525043 3859 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:31.525104 3859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:31.525123 3859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:31.525141 3859 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:31.525156 3859 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:31.526856 3859 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:37945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:46661
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:46661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3859
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:17:31.527155 3859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:31.527416 3859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:31.532548 3869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:31.532548 3871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:31.532724 3859 server_base.cc:1034] running on GCE node
W20260811 05:17:31.532557 3878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:31.533126 3859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:31.533449 3859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:31.534614 3859 hybrid_clock.cc:648] HybridClock initialized: now 1786425451534586 us; error 41 us; skew 500 ppm
Aug 11 05:17:31 dist-test-slave-f7th krb5kdc[3832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425451, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:31.540465 3859 init.cc:377] Logged in from keytab as kudu/127.28.143.190@KRBTEST.COM (short username kudu)
I20260811 05:17:31.541044 3859 webserver.cc:460] Webserver started at http://127.28.143.190:40873/ using document root <none> and password file <none>
I20260811 05:17:31.541296 3859 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:31.541348 3859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:31.541489 3859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:31.542605 3859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "7d349dccec534d05a8c2d05515549833"
format_stamp: "Formatted at 2026-08-11 05:17:31 on dist-test-slave-f7th"
server_key: "b99d229c20d797566bdc6d4ef6c5ecc0"
server_key_iv: "84a80c89c8033e57d60681ace5f3945a"
server_key_version: "encryptionkey@0"
I20260811 05:17:31.542958 3859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "7d349dccec534d05a8c2d05515549833"
format_stamp: "Formatted at 2026-08-11 05:17:31 on dist-test-slave-f7th"
server_key: "b99d229c20d797566bdc6d4ef6c5ecc0"
server_key_iv: "84a80c89c8033e57d60681ace5f3945a"
server_key_version: "encryptionkey@0"
I20260811 05:17:31.544343 3859 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 05:17:31.545279 3894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:31.545475 3859 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 05:17:31.545562 3859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "7d349dccec534d05a8c2d05515549833"
format_stamp: "Formatted at 2026-08-11 05:17:31 on dist-test-slave-f7th"
server_key: "b99d229c20d797566bdc6d4ef6c5ecc0"
server_key_iv: "84a80c89c8033e57d60681ace5f3945a"
server_key_version: "encryptionkey@0"
I20260811 05:17:31.545632 3859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:31.570876 3859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:31.573315 3859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:31.573509 3859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:31.577651 3859 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:46661
I20260811 05:17:31.577692 3986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:46661 every 8 connection(s)
I20260811 05:17:31.578049 3859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:17:31.578994 3988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:31.581632 3988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d349dccec534d05a8c2d05515549833: Bootstrap starting.
I20260811 05:17:31.582285 3988 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d349dccec534d05a8c2d05515549833: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:31.582558 3988 log.cc:824] T 00000000000000000000000000000000 P 7d349dccec534d05a8c2d05515549833: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:31.583330 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 3859
I20260811 05:17:31.583431 3988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d349dccec534d05a8c2d05515549833: No bootstrap required, opened a new log
I20260811 05:17:31.583449 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:17:31.583621 29246 external_mini_cluster.cc:1428] Setting key 93b708b60afdbd7c41f64764dcefc6ea
I20260811 05:17:31.585450 3988 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d349dccec534d05a8c2d05515549833 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d349dccec534d05a8c2d05515549833" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 46661 } }
I20260811 05:17:31.585626 3988 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d349dccec534d05a8c2d05515549833 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:31.585665 3988 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d349dccec534d05a8c2d05515549833 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d349dccec534d05a8c2d05515549833, State: Initialized, Role: FOLLOWER
I20260811 05:17:31.585842 3988 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d349dccec534d05a8c2d05515549833 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d349dccec534d05a8c2d05515549833" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 46661 } }
I20260811 05:17:31.585935 3988 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d349dccec534d05a8c2d05515549833 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:17:31.585986 3988 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d349dccec534d05a8c2d05515549833 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:17:31.586040 3988 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d349dccec534d05a8c2d05515549833 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:31.587162 3988 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d349dccec534d05a8c2d05515549833 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d349dccec534d05a8c2d05515549833" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 46661 } }
I20260811 05:17:31.587263 3988 leader_election.cc:302] T 00000000000000000000000000000000 P 7d349dccec534d05a8c2d05515549833 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d349dccec534d05a8c2d05515549833; no voters:
I20260811 05:17:31.587425 3988 leader_election.cc:288] T 00000000000000000000000000000000 P 7d349dccec534d05a8c2d05515549833 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:17:31.587478 3997 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d349dccec534d05a8c2d05515549833 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:31.587692 3997 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d349dccec534d05a8c2d05515549833 [term 1 LEADER]: Becoming Leader. State: Replica: 7d349dccec534d05a8c2d05515549833, State: Running, Role: LEADER
I20260811 05:17:31.587713 3988 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d349dccec534d05a8c2d05515549833 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:31.587859 3997 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d349dccec534d05a8c2d05515549833 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d349dccec534d05a8c2d05515549833" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 46661 } }
I20260811 05:17:31.588266 3999 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d349dccec534d05a8c2d05515549833 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d349dccec534d05a8c2d05515549833" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d349dccec534d05a8c2d05515549833" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 46661 } } }
I20260811 05:17:31.588346 3999 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d349dccec534d05a8c2d05515549833 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:31.588675 3998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d349dccec534d05a8c2d05515549833 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d349dccec534d05a8c2d05515549833. Latest consensus state: current_term: 1 leader_uuid: "7d349dccec534d05a8c2d05515549833" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d349dccec534d05a8c2d05515549833" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 46661 } } }
I20260811 05:17:31.588826 3998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d349dccec534d05a8c2d05515549833 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:31.588987 4010 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:17:31.589586 4010 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:17:31.591113 4010 catalog_manager.cc:1347] Generated new cluster ID: fa54acd4d3fd405abbb7b7c594e4c670
I20260811 05:17:31.591159 4010 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 05:17:31 dist-test-slave-f7th krb5kdc[3832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425451, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:17:31.596057 3996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:31.584181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46285 (local address 127.28.143.190:46661)
0811 05:17:31.584360 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:31.584369 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:31.584401 (+ 32us) server_negotiation.cc:407] Connection header received
0811 05:17:31.585419 (+ 1018us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:31.585430 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:31.585491 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:31.585621 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:31.586449 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:31.587490 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:31.588592 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:31.589218 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:31.589321 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:31.592231 (+ 2910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:31.592235 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:31.592236 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:31.592249 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:31.594283 (+ 2034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:31.594728 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:31.594732 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:31.594732 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:31.594865 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:31.595084 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:31.595085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:31.595085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:31.595387 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:31.595544 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:31.595697 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 05:17:31.595836 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":85,"threads_started":1}
I20260811 05:17:31.597049 29246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:17:31.605962 3996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:31.598736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51345 (local address 127.28.143.190:46661)
0811 05:17:31.598790 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:31.598792 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:31.598849 (+ 57us) server_negotiation.cc:407] Connection header received
0811 05:17:31.598921 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:31.598922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:31.598934 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:31.598995 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:31.599626 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:31.600402 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:31.601415 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:31.601755 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:31.601824 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:31.603181 (+ 1357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:31.603183 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:31.603183 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:31.603193 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:31.604823 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:31.605200 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:31.605201 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:31.605202 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:31.605266 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:31.605433 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:31.605434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:31.605434 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:31.605571 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:31.605665 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:31.605770 (+ 105us) server_negotiation.cc:299] Negotiation successful
0811 05:17:31.605852 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 05:17:31.607717 4010 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:17:31.608253 4010 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:17:31.614308 4010 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d349dccec534d05a8c2d05515549833: Generated new TSK 0
I20260811 05:17:31.614524 4010 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:17:31.618876 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 3859
2026-08-11T05:17:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1549 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425159060450-29246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:17:31 dist-test-slave-f7th krb5kdc[4045](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:17:31 dist-test-slave-f7th krb5kdc[4045](info): set up 2 sockets
Aug 11 05:17:31 dist-test-slave-f7th krb5kdc[4045](info): commencing operation
krb5kdc: starting...
W20260811 05:17:32.841759 29246 mini_kdc.cc:121] Time spent starting KDC: real 1.186s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Aug 11 05:17:32 dist-test-slave-f7th krb5kdc[4045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:17:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:17:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
I20260811 05:17:32.981272 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:40797
--webserver_interface=127.28.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:40445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:40797
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:33.069320 4080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:33.069548 4080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:33.069582 4080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:33.070791 4080 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:33.070839 4080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:33.070868 4080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:33.070883 4080 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:33.070899 4080 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:33.072484 4080 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:40445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:40797
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:40797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:17:33.072777 4080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:33.073009 4080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:33.078020 4097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:33.078086 4092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:33.078177 4102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:33.078202 4080 server_base.cc:1034] running on GCE node
I20260811 05:17:33.078452 4080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:33.078787 4080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:33.079962 4080 hybrid_clock.cc:648] HybridClock initialized: now 1786425453079942 us; error 37 us; skew 500 ppm
Aug 11 05:17:33 dist-test-slave-f7th krb5kdc[4045](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425453, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:33.082826 4080 init.cc:377] Logged in from keytab as kudu/127.28.143.190@KRBTEST.COM (short username kudu)
I20260811 05:17:33.083325 4080 webserver.cc:460] Webserver started at http://127.28.143.190:43615/ using document root <none> and password file <none>
I20260811 05:17:33.083566 4080 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:33.083617 4080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:33.083751 4080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:33.084908 4080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "ee7fb7e84fa1472a9c67864abd4909c8"
format_stamp: "Formatted at 2026-08-11 05:17:33 on dist-test-slave-f7th"
server_key: "562342111ebed2b36e42fe9868278ba5"
server_key_iv: "42728d108718bb81abc8d59b9c64e19b"
server_key_version: "encryptionkey@0"
I20260811 05:17:33.085248 4080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "ee7fb7e84fa1472a9c67864abd4909c8"
format_stamp: "Formatted at 2026-08-11 05:17:33 on dist-test-slave-f7th"
server_key: "562342111ebed2b36e42fe9868278ba5"
server_key_iv: "42728d108718bb81abc8d59b9c64e19b"
server_key_version: "encryptionkey@0"
I20260811 05:17:33.086582 4080 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 05:17:33.087342 4110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:33.087495 4080 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 05:17:33.087597 4080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "ee7fb7e84fa1472a9c67864abd4909c8"
format_stamp: "Formatted at 2026-08-11 05:17:33 on dist-test-slave-f7th"
server_key: "562342111ebed2b36e42fe9868278ba5"
server_key_iv: "42728d108718bb81abc8d59b9c64e19b"
server_key_version: "encryptionkey@0"
I20260811 05:17:33.087658 4080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:33.102326 4080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:33.104552 4080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:33.104740 4080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:33.108795 4080 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:40797
I20260811 05:17:33.108837 4196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:40797 every 8 connection(s)
I20260811 05:17:33.109273 4080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:17:33.110272 4199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:33.112697 4199 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee7fb7e84fa1472a9c67864abd4909c8: Bootstrap starting.
I20260811 05:17:33.113322 4199 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee7fb7e84fa1472a9c67864abd4909c8: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:33.113586 4199 log.cc:824] T 00000000000000000000000000000000 P ee7fb7e84fa1472a9c67864abd4909c8: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:33.114426 4199 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee7fb7e84fa1472a9c67864abd4909c8: No bootstrap required, opened a new log
I20260811 05:17:33.116276 4199 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee7fb7e84fa1472a9c67864abd4909c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee7fb7e84fa1472a9c67864abd4909c8" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 40797 } }
I20260811 05:17:33.116441 4199 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee7fb7e84fa1472a9c67864abd4909c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:33.116464 4199 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee7fb7e84fa1472a9c67864abd4909c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee7fb7e84fa1472a9c67864abd4909c8, State: Initialized, Role: FOLLOWER
I20260811 05:17:33.116648 4199 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee7fb7e84fa1472a9c67864abd4909c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee7fb7e84fa1472a9c67864abd4909c8" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 40797 } }
I20260811 05:17:33.116722 4199 raft_consensus.cc:388] T 00000000000000000000000000000000 P ee7fb7e84fa1472a9c67864abd4909c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:17:33.116748 4199 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee7fb7e84fa1472a9c67864abd4909c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:17:33.116799 4199 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee7fb7e84fa1472a9c67864abd4909c8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:33.117728 4199 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee7fb7e84fa1472a9c67864abd4909c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee7fb7e84fa1472a9c67864abd4909c8" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 40797 } }
I20260811 05:17:33.117825 4199 leader_election.cc:302] T 00000000000000000000000000000000 P ee7fb7e84fa1472a9c67864abd4909c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee7fb7e84fa1472a9c67864abd4909c8; no voters:
I20260811 05:17:33.118018 4199 leader_election.cc:288] T 00000000000000000000000000000000 P ee7fb7e84fa1472a9c67864abd4909c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:17:33.118055 4205 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee7fb7e84fa1472a9c67864abd4909c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:33.118119 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 4080
I20260811 05:17:33.118185 4205 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee7fb7e84fa1472a9c67864abd4909c8 [term 1 LEADER]: Becoming Leader. State: Replica: ee7fb7e84fa1472a9c67864abd4909c8, State: Running, Role: LEADER
I20260811 05:17:33.118243 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:17:33.118332 4205 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee7fb7e84fa1472a9c67864abd4909c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee7fb7e84fa1472a9c67864abd4909c8" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 40797 } }
I20260811 05:17:33.118342 4199 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee7fb7e84fa1472a9c67864abd4909c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:33.118528 29246 external_mini_cluster.cc:1428] Setting key 7c09683b3494f8994468d4b2420da18f
I20260811 05:17:33.118821 4207 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee7fb7e84fa1472a9c67864abd4909c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ee7fb7e84fa1472a9c67864abd4909c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee7fb7e84fa1472a9c67864abd4909c8" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 40797 } } }
I20260811 05:17:33.118798 4209 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee7fb7e84fa1472a9c67864abd4909c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee7fb7e84fa1472a9c67864abd4909c8. Latest consensus state: current_term: 1 leader_uuid: "ee7fb7e84fa1472a9c67864abd4909c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee7fb7e84fa1472a9c67864abd4909c8" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 40797 } } }
I20260811 05:17:33.118924 4207 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee7fb7e84fa1472a9c67864abd4909c8 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:33.118930 4209 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee7fb7e84fa1472a9c67864abd4909c8 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:33.119160 4213 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:17:33.120085 4213 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:17:33.122062 4213 catalog_manager.cc:1347] Generated new cluster ID: abda6860668c47b8bb0b18ca14d3d667
I20260811 05:17:33.122119 4213 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:17:33.125108 4214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:33.119235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49735 (local address 127.28.143.190:40797)
0811 05:17:33.119398 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:33.119408 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:33.119750 (+ 342us) server_negotiation.cc:407] Connection header received
0811 05:17:33.119938 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:33.119946 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:33.120012 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:33.120165 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:33.120535 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:33.121693 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:33.122819 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:33.123484 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:33.123610 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:33.124303 (+ 693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:33.124307 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:33.124307 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:33.124522 (+ 215us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:33.124561 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:33.124591 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:33.124692 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:33.124812 (+ 120us) server_negotiation.cc:299] Negotiation successful
0811 05:17:33.124886 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":80,"threads_started":1}
I20260811 05:17:33.126384 29246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:17:33.131405 4214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:33.127996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57443 (local address 127.28.143.190:40797)
0811 05:17:33.128031 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:33.128033 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:33.128040 (+ 7us) server_negotiation.cc:407] Connection header received
0811 05:17:33.128061 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:33.128061 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:33.128072 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:33.128132 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:33.128232 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:33.129056 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:33.130057 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:33.130392 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:33.130467 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:33.131157 (+ 690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:33.131158 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:33.131159 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:33.131163 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:33.131180 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:33.131185 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:33.131236 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:33.131294 (+ 58us) server_negotiation.cc:299] Negotiation successful
0811 05:17:33.131322 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260811 05:17:33.131749 4213 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:17:33.132274 4213 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:17:33.139453 4213 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ee7fb7e84fa1472a9c67864abd4909c8: Generated new TSK 0
I20260811 05:17:33.139658 4213 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:17:33.146611 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 4080
2026-08-11T05:17:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1541 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425159060450-29246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:17:33 dist-test-slave-f7th krb5kdc[4261](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:17:33 dist-test-slave-f7th krb5kdc[4261](info): set up 2 sockets
Aug 11 05:17:33 dist-test-slave-f7th krb5kdc[4261](info): commencing operation
krb5kdc: starting...
W20260811 05:17:34.405200 29246 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Aug 11 05:17:34 dist-test-slave-f7th krb5kdc[4261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:17:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:17:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
I20260811 05:17:34.541620 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:40177
--webserver_interface=127.28.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:45467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:40177
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:34.629030 4281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:34.629264 4281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:34.629299 4281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:34.630537 4281 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:34.630647 4281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:34.630709 4281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:34.630740 4281 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:34.630759 4281 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:34.632715 4281 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:45467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:40177
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:40177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:17:34.633008 4281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:34.633244 4281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:34.638311 4289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:34.638312 4300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:34.638484 4281 server_base.cc:1034] running on GCE node
W20260811 05:17:34.638345 4294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:34.638866 4281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:34.639168 4281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:34.640344 4281 hybrid_clock.cc:648] HybridClock initialized: now 1786425454640325 us; error 36 us; skew 500 ppm
Aug 11 05:17:34 dist-test-slave-f7th krb5kdc[4261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425454, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:34.643160 4281 init.cc:377] Logged in from keytab as kudu/127.28.143.190@KRBTEST.COM (short username kudu)
I20260811 05:17:34.643712 4281 webserver.cc:460] Webserver started at http://127.28.143.190:38749/ using document root <none> and password file <none>
I20260811 05:17:34.643939 4281 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:34.643999 4281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:34.644134 4281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:34.645323 4281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "1a1c77f285ad48df8acd522a1dfdb51c"
format_stamp: "Formatted at 2026-08-11 05:17:34 on dist-test-slave-f7th"
server_key: "38108662482a17da870ce3e92c6f75ae"
server_key_iv: "a439f6e9ba23d3de6842957d3fb8a563"
server_key_version: "encryptionkey@0"
I20260811 05:17:34.645694 4281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "1a1c77f285ad48df8acd522a1dfdb51c"
format_stamp: "Formatted at 2026-08-11 05:17:34 on dist-test-slave-f7th"
server_key: "38108662482a17da870ce3e92c6f75ae"
server_key_iv: "a439f6e9ba23d3de6842957d3fb8a563"
server_key_version: "encryptionkey@0"
I20260811 05:17:34.647113 4281 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 05:17:34.648087 4312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:34.648319 4281 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:17:34.648422 4281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "1a1c77f285ad48df8acd522a1dfdb51c"
format_stamp: "Formatted at 2026-08-11 05:17:34 on dist-test-slave-f7th"
server_key: "38108662482a17da870ce3e92c6f75ae"
server_key_iv: "a439f6e9ba23d3de6842957d3fb8a563"
server_key_version: "encryptionkey@0"
I20260811 05:17:34.648511 4281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:34.652040 4281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:34.654534 4281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:34.654708 4281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:34.658902 4281 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:40177
I20260811 05:17:34.659013 4412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:40177 every 8 connection(s)
I20260811 05:17:34.659308 4281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:17:34.660362 4414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:34.663141 4414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a1c77f285ad48df8acd522a1dfdb51c: Bootstrap starting.
I20260811 05:17:34.663826 4414 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a1c77f285ad48df8acd522a1dfdb51c: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:34.664124 4414 log.cc:824] T 00000000000000000000000000000000 P 1a1c77f285ad48df8acd522a1dfdb51c: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:34.664971 4414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a1c77f285ad48df8acd522a1dfdb51c: No bootstrap required, opened a new log
I20260811 05:17:34.666654 4414 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a1c77f285ad48df8acd522a1dfdb51c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a1c77f285ad48df8acd522a1dfdb51c" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 40177 } }
I20260811 05:17:34.666816 4414 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a1c77f285ad48df8acd522a1dfdb51c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:34.666857 4414 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a1c77f285ad48df8acd522a1dfdb51c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a1c77f285ad48df8acd522a1dfdb51c, State: Initialized, Role: FOLLOWER
I20260811 05:17:34.667004 4414 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a1c77f285ad48df8acd522a1dfdb51c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a1c77f285ad48df8acd522a1dfdb51c" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 40177 } }
I20260811 05:17:34.667080 4414 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a1c77f285ad48df8acd522a1dfdb51c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:17:34.667114 4414 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a1c77f285ad48df8acd522a1dfdb51c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:17:34.667167 4414 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a1c77f285ad48df8acd522a1dfdb51c [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:34.667689 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 4281
I20260811 05:17:34.667824 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:17:34.668069 29246 external_mini_cluster.cc:1428] Setting key 123aac4862003df0ad26c9c306455f84
I20260811 05:17:34.668370 4414 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a1c77f285ad48df8acd522a1dfdb51c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a1c77f285ad48df8acd522a1dfdb51c" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 40177 } }
I20260811 05:17:34.668484 4414 leader_election.cc:302] T 00000000000000000000000000000000 P 1a1c77f285ad48df8acd522a1dfdb51c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a1c77f285ad48df8acd522a1dfdb51c; no voters:
I20260811 05:17:34.668771 4414 leader_election.cc:288] T 00000000000000000000000000000000 P 1a1c77f285ad48df8acd522a1dfdb51c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:17:34.668812 4419 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a1c77f285ad48df8acd522a1dfdb51c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:34.669006 4419 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a1c77f285ad48df8acd522a1dfdb51c [term 1 LEADER]: Becoming Leader. State: Replica: 1a1c77f285ad48df8acd522a1dfdb51c, State: Running, Role: LEADER
I20260811 05:17:34.669126 4414 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a1c77f285ad48df8acd522a1dfdb51c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:34.669155 4419 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a1c77f285ad48df8acd522a1dfdb51c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a1c77f285ad48df8acd522a1dfdb51c" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 40177 } }
I20260811 05:17:34.669575 4425 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a1c77f285ad48df8acd522a1dfdb51c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a1c77f285ad48df8acd522a1dfdb51c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a1c77f285ad48df8acd522a1dfdb51c" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 40177 } } }
I20260811 05:17:34.669593 4426 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a1c77f285ad48df8acd522a1dfdb51c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a1c77f285ad48df8acd522a1dfdb51c. Latest consensus state: current_term: 1 leader_uuid: "1a1c77f285ad48df8acd522a1dfdb51c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a1c77f285ad48df8acd522a1dfdb51c" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 40177 } } }
I20260811 05:17:34.669754 4425 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a1c77f285ad48df8acd522a1dfdb51c [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:34.669764 4426 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a1c77f285ad48df8acd522a1dfdb51c [sys.catalog]: This master's current role is: LEADER
W20260811 05:17:34.671000 4446 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a1c77f285ad48df8acd522a1dfdb51c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:17:34.671058 4446 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:17:34.671226 4436 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:17:34.671546 4436 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:17:34.671576 4421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:34.668701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35451 (local address 127.28.143.190:40177)
0811 05:17:34.669569 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:34.669580 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:34.669620 (+ 40us) server_negotiation.cc:407] Connection header received
0811 05:17:34.669752 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:34.669760 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:34.669793 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:34.670264 (+ 471us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:34.671055 (+ 791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:34.671059 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:34.671059 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:34.671064 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:34.671120 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:34.671135 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:34.671199 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:34.671335 (+ 136us) server_negotiation.cc:299] Negotiation successful
0811 05:17:34.671386 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":807,"thread_start_us":105,"threads_started":1}
I20260811 05:17:34.672480 29246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:17:34.673424 4436 catalog_manager.cc:1347] Generated new cluster ID: 964ddf90bc414bfeade602a27ebc910e
I20260811 05:17:34.673477 4436 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:17:34.678032 4421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:34.673912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60443 (local address 127.28.143.190:40177)
0811 05:17:34.673953 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:34.673954 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:34.677000 (+ 3046us) server_negotiation.cc:407] Connection header received
0811 05:17:34.677133 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:34.677134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:34.677158 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:34.677212 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:34.677818 (+ 606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:34.677820 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:34.677820 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:34.677825 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:34.677841 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:34.677846 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:34.677895 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:34.677941 (+ 46us) server_negotiation.cc:299] Negotiation successful
0811 05:17:34.677956 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 05:17:34.680904 4436 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:17:34.681099 4436 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:17:34.698812 4436 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a1c77f285ad48df8acd522a1dfdb51c: Generated new TSK 0
I20260811 05:17:34.699143 4436 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:17:34.703472 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 4281
2026-08-11T05:17:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1546 ms)
[----------] 18 tests from AuthTokenIssuingTest (27558 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:17:34 dist-test-slave-f7th krb5kdc[4462](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:17:34 dist-test-slave-f7th krb5kdc[4462](info): set up 2 sockets
Aug 11 05:17:34 dist-test-slave-f7th krb5kdc[4462](info): commencing operation
krb5kdc: starting...
W20260811 05:17:35.940004 29246 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Aug 11 05:17:36 dist-test-slave-f7th krb5kdc[4462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:17:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:17:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
I20260811 05:17:36.076740 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:38881
--webserver_interface=127.28.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:45007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:38881,127.28.143.189:44537,127.28.143.188:41587
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:36.166298 4487 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 05:17:36.166565 4487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:36.166594 4487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:36.166623 4487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:36.167923 4487 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:36.167977 4487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:36.167996 4487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:36.168013 4487 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:36.168028 4487 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:36.169735 4487 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:45007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:38881,127.28.143.189:44537,127.28.143.188:41587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:38881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:17:36.170053 4487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:36.170296 4487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:36.175788 4504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:36.175762 4496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:36.175802 4487 server_base.cc:1034] running on GCE node
W20260811 05:17:36.175766 4497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:36.176308 4487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:36.176684 4487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:36.177856 4487 hybrid_clock.cc:648] HybridClock initialized: now 1786425456177835 us; error 41 us; skew 500 ppm
Aug 11 05:17:36 dist-test-slave-f7th krb5kdc[4462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425456, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:36.180455 4487 init.cc:377] Logged in from keytab as kudu/127.28.143.190@KRBTEST.COM (short username kudu)
I20260811 05:17:36.180958 4487 webserver.cc:460] Webserver started at http://127.28.143.190:36691/ using document root <none> and password file <none>
I20260811 05:17:36.181208 4487 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:36.181262 4487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:36.181399 4487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:36.182549 4487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "cc6daa2d3811400eb1cc9a35bd938de6"
format_stamp: "Formatted at 2026-08-11 05:17:36 on dist-test-slave-f7th"
server_key: "0e5b7e6e63e44078868b631f9dfc43f0"
server_key_iv: "f203c3daa16feae50bb456f34afc4fad"
server_key_version: "encryptionkey@0"
I20260811 05:17:36.182916 4487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "cc6daa2d3811400eb1cc9a35bd938de6"
format_stamp: "Formatted at 2026-08-11 05:17:36 on dist-test-slave-f7th"
server_key: "0e5b7e6e63e44078868b631f9dfc43f0"
server_key_iv: "f203c3daa16feae50bb456f34afc4fad"
server_key_version: "encryptionkey@0"
I20260811 05:17:36.184285 4487 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 05:17:36.185128 4518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:36.185288 4487 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 05:17:36.185370 4487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "cc6daa2d3811400eb1cc9a35bd938de6"
format_stamp: "Formatted at 2026-08-11 05:17:36 on dist-test-slave-f7th"
server_key: "0e5b7e6e63e44078868b631f9dfc43f0"
server_key_iv: "f203c3daa16feae50bb456f34afc4fad"
server_key_version: "encryptionkey@0"
I20260811 05:17:36.185438 4487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:36.203665 4487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:36.206099 4487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:36.206257 4487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:36.210460 4487 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:38881
I20260811 05:17:36.210507 4624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:38881 every 8 connection(s)
I20260811 05:17:36.210863 4487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:17:36.211808 4625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:36.213183 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 4487
I20260811 05:17:36.213287 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:17:36.213459 29246 external_mini_cluster.cc:1428] Setting key 2471544449ce6a52aca14935b7d669da
I20260811 05:17:36.213572 4625 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.143.190" port: 38881 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 05:17:36 dist-test-slave-f7th krb5kdc[4462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425456, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.28.143.189@KRBTEST.COM; defaulting to no policy
I20260811 05:17:36.224416 4627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.214290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:38881 (local address 127.0.0.1:55440)
0811 05:17:36.214486 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:36.214536 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:36.215169 (+ 633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:36.215528 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:36.215530 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:36.215805 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:36.215974 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:36.215978 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.217224 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.217225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:36.218266 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:36.218269 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.218797 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.218798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:36.218886 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:36.219423 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:17:36.219435 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:36.221151 (+ 1716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:36.222851 (+ 1700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:36.222857 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:36.222860 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:36.223049 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:36.223258 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:36.223259 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:36.223259 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:36.223350 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:36.223773 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:36.223781 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:36.223908 (+ 127us) client_negotiation.cc:769] Sending connection context
0811 05:17:36.224024 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 05:17:36.224268 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":103,"thread_start_us":100,"threads_started":1}
I20260811 05:17:36.224460 4632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.214352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55440 (local address 127.28.143.190:38881)
0811 05:17:36.214511 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:36.214520 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:36.214632 (+ 112us) server_negotiation.cc:407] Connection header received
0811 05:17:36.215381 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:36.215384 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:36.215410 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:36.215519 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:36.216067 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.217128 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.218362 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.218707 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.218793 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:36.221238 (+ 2445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:36.221246 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:36.221247 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:36.221254 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:36.222773 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:36.223120 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:36.223124 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:36.223124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:36.223202 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:36.223440 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:36.223441 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:36.223442 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:36.223684 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:36.223756 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:36.224073 (+ 317us) server_negotiation.cc:299] Negotiation successful
0811 05:17:36.224205 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":125,"thread_start_us":60,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.189@KRBTEST.COM" created.
I20260811 05:17:36.225204 4625 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.143.189" port: 44537 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:17:36.225639 4627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.225486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.189:44537 (local address 127.0.0.1:58402)
0811 05:17:36.225556 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:36.225611 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.189:44537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 05:17:36.225821 4526 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.143.189:44537: connect: Connection refused (error 111)
I20260811 05:17:36.226300 4627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.226199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.189:44537 (local address 127.0.0.1:58408)
0811 05:17:36.226245 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:36.226278 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.189:44537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260811 05:17:36.226408 4625 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.143.189:44537: Network error: Client connection negotiation failed: client connection to 127.28.143.189:44537: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.189.keytab.
Entry for principal kudu/127.28.143.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.189.keytab.
I20260811 05:17:36.244722 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.189:44537
--webserver_interface=127.28.143.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.189.keytab
--principal=kudu/127.28.143.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:45007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:38881,127.28.143.189:44537,127.28.143.188:41587
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:17:36.275671 4625 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.143.189" port: 44537 } attempt: 1
I20260811 05:17:36.276226 4627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.276098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.189:44537 (local address 127.0.0.1:58418)
0811 05:17:36.276159 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:36.276196 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.189:44537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 05:17:36.276607 4627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.276517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.189:44537 (local address 127.0.0.1:58424)
0811 05:17:36.276562 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:36.276587 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.189:44537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260811 05:17:36.276712 4625 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.143.189:44537: Network error: Client connection negotiation failed: client connection to 127.28.143.189:44537: connect: Connection refused (error 111)
W20260811 05:17:36.330163 4640 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 05:17:36.330408 4640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:36.330425 4640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:36.330448 4640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:36.331734 4640 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:36.331795 4640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:36.331808 4640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:36.331820 4640 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:36.331831 4640 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:36.333554 4640 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:45007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:38881,127.28.143.189:44537,127.28.143.188:41587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.189.keytab
--principal=kudu/127.28.143.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.189:44537
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.28.143.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:17:36.333973 4640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:36.334241 4640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:36.339982 4650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:36.339985 4649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:36.339988 4654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:36.340029 4640 server_base.cc:1034] running on GCE node
I20260811 05:17:36.340632 4640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:36.341089 4640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:36.342273 4640 hybrid_clock.cc:648] HybridClock initialized: now 1786425456342238 us; error 53 us; skew 500 ppm
Aug 11 05:17:36 dist-test-slave-f7th krb5kdc[4462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425456, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:36.344877 4625 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.143.189" port: 44537 } attempt: 2
I20260811 05:17:36.345368 4627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.345226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.189:44537 (local address 127.0.0.1:58432)
0811 05:17:36.345287 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:36.345330 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.189:44537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260811 05:17:36.345338 4640 init.cc:377] Logged in from keytab as kudu/127.28.143.189@KRBTEST.COM (short username kudu)
I20260811 05:17:36.345773 4627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.345671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.189:44537 (local address 127.0.0.1:58436)
0811 05:17:36.345717 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:36.345748 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.189:44537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260811 05:17:36.345942 4625 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.143.189:44537: Network error: Client connection negotiation failed: client connection to 127.28.143.189:44537: connect: Connection refused (error 111)
I20260811 05:17:36.345973 4640 webserver.cc:460] Webserver started at http://127.28.143.189:33059/ using document root <none> and password file <none>
I20260811 05:17:36.346282 4640 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:36.346377 4640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:36.346504 4640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:36.348145 4640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-1/data/instance:
uuid: "6ff0f0c96b694fa696b89c27e396b31c"
format_stamp: "Formatted at 2026-08-11 05:17:36 on dist-test-slave-f7th"
server_key: "eefe89fd40f9e035bbd8aebd4df23590"
server_key_iv: "f7f41bdb24baf637bd260902a3080f9e"
server_key_version: "encryptionkey@0"
I20260811 05:17:36.348517 4640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-1/wal/instance:
uuid: "6ff0f0c96b694fa696b89c27e396b31c"
format_stamp: "Formatted at 2026-08-11 05:17:36 on dist-test-slave-f7th"
server_key: "eefe89fd40f9e035bbd8aebd4df23590"
server_key_iv: "f7f41bdb24baf637bd260902a3080f9e"
server_key_version: "encryptionkey@0"
I20260811 05:17:36.350194 4640 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 05:17:36.351099 4670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:36.351302 4640 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:17:36.351388 4640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-1/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-1/wal
uuid: "6ff0f0c96b694fa696b89c27e396b31c"
format_stamp: "Formatted at 2026-08-11 05:17:36 on dist-test-slave-f7th"
server_key: "eefe89fd40f9e035bbd8aebd4df23590"
server_key_iv: "f7f41bdb24baf637bd260902a3080f9e"
server_key_version: "encryptionkey@0"
I20260811 05:17:36.351483 4640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:36.371220 4640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:36.373729 4640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:36.373858 4640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:36.377986 4640 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.189:44537
I20260811 05:17:36.378062 4772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.189:44537 every 8 connection(s)
I20260811 05:17:36.378438 4640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-1/data/info.pb
I20260811 05:17:36.379403 4773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:36.381062 4773 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.143.190" port: 38881 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:17:36.381323 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 4640
I20260811 05:17:36.381420 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-1/wal/instance
I20260811 05:17:36.381603 29246 external_mini_cluster.cc:1428] Setting key c4d4a3d76ad3ca1f91f2849767d81fba
Aug 11 05:17:36 dist-test-slave-f7th krb5kdc[4462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425456, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.189@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.28.143.188@KRBTEST.COM; defaulting to no policy
I20260811 05:17:36.392192 4632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.382018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55454 (local address 127.28.143.190:38881)
0811 05:17:36.382083 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:36.382086 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:36.382337 (+ 251us) server_negotiation.cc:407] Connection header received
0811 05:17:36.383020 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:36.383023 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:36.383043 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:36.383095 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:36.384030 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.384795 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.385939 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.386249 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.386340 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:36.389144 (+ 2804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:36.389146 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:36.389146 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:36.389154 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:36.390653 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:36.391176 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:36.391177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:36.391177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:36.391240 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:36.391457 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:36.391458 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:36.391458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:36.391570 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:36.391619 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:36.391943 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 05:17:36.392097 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 05:17:36.392310 4775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.381982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:38881 (local address 127.0.0.1:55454)
0811 05:17:36.382185 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:36.382237 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:36.382871 (+ 634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:36.383215 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:36.383221 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:36.383615 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:36.383901 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:36.383907 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.384924 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.384926 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:36.385832 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:36.385835 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.386361 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.386362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:36.386484 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:36.387101 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:17:36.387113 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:36.389009 (+ 1896us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:36.390748 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:36.390757 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:36.390760 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:36.391098 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:36.391299 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:36.391300 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:36.391300 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:36.391416 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:36.391653 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:36.391661 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:36.391807 (+ 146us) client_negotiation.cc:769] Sending connection context
0811 05:17:36.391909 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 05:17:36.392110 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":109,"threads_started":1}
I20260811 05:17:36.392976 4773 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.143.189" port: 44537 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.188@KRBTEST.COM" created.
Aug 11 05:17:36 dist-test-slave-f7th krb5kdc[4462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425456, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.189@KRBTEST.COM for kudu/127.28.143.189@KRBTEST.COM
I20260811 05:17:36.401476 4775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.393284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.189:44537 (local address 127.0.0.1:58448)
0811 05:17:36.393360 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:36.393372 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:36.393424 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:36.393750 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:36.393751 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:36.393881 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:36.393975 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:36.393977 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.394927 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.394928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:36.395589 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:36.395592 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.396201 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.396202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:36.396287 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:36.396852 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:17:36.396857 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:36.398240 (+ 1383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:36.400370 (+ 2130us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:36.400371 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:36.400372 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:36.400541 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:36.400795 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:36.400796 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:36.400797 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:36.400847 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:36.401138 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:36.401139 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:36.401179 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 05:17:36.401240 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 05:17:36.401359 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 05:17:36.401510 4777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.393434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58448 (local address 127.28.143.189:44537)
0811 05:17:36.393610 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:36.393615 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:36.393636 (+ 21us) server_negotiation.cc:407] Connection header received
0811 05:17:36.393655 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:36.393656 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:36.393668 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:36.393732 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:36.394074 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.394815 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.395688 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.396036 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.396179 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:36.398343 (+ 2164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:36.398356 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:36.398356 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:36.398365 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:36.400271 (+ 1906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:36.400665 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:36.400670 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:36.400670 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:36.400733 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:36.400900 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:36.400900 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:36.400901 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:36.401075 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:36.401154 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:36.401280 (+ 126us) server_negotiation.cc:299] Negotiation successful
0811 05:17:36.401413 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":144,"thread_start_us":96,"threads_started":1}
I20260811 05:17:36.402160 4773 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.143.188" port: 41587 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:17:36.402546 4775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.402425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:41587 (local address 127.0.0.1:48118)
0811 05:17:36.402469 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:36.402519 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.188:41587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260811 05:17:36.402710 4672 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.143.188:41587: connect: Connection refused (error 111)
I20260811 05:17:36.403306 4775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.403177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:41587 (local address 127.0.0.1:48128)
0811 05:17:36.403249 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:36.403282 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.188:41587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 05:17:36.403434 4773 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.143.188:41587: Network error: Client connection negotiation failed: client connection to 127.28.143.188:41587: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.188.keytab.
Entry for principal kudu/127.28.143.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.188.keytab.
I20260811 05:17:36.413945 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.188:41587
--webserver_interface=127.28.143.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.188.keytab
--principal=kudu/127.28.143.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:45007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:38881,127.28.143.189:44537,127.28.143.188:41587
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:17:36.437098 4625 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.143.189" port: 44537 } attempt: 3
Aug 11 05:17:36 dist-test-slave-f7th krb5kdc[4462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425456, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for kudu/127.28.143.189@KRBTEST.COM
I20260811 05:17:36.445328 4777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.437648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58460 (local address 127.28.143.189:44537)
0811 05:17:36.437706 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:36.437708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:36.437718 (+ 10us) server_negotiation.cc:407] Connection header received
0811 05:17:36.437784 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:36.437785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:36.437796 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:36.437845 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:36.438230 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.438760 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.439912 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.440155 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.440226 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:36.442550 (+ 2324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:36.442553 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:36.442553 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:36.442562 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:36.444058 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:36.444501 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:36.444503 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:36.444503 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:36.444583 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:36.444843 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:36.444844 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:36.444844 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:36.444964 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:36.445011 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:36.445176 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 05:17:36.445257 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 05:17:36.445360 4627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.437548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.189:44537 (local address 127.0.0.1:58460)
0811 05:17:36.437634 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:36.437648 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:36.437715 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:36.437866 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:36.437867 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:36.438073 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:36.438157 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:36.438160 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.438847 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.438848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:36.439781 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:36.439787 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.440238 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.440239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:36.440316 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:36.440856 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:17:36.440862 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:36.442414 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:36.444160 (+ 1746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:36.444161 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:36.444162 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:36.444395 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:36.444700 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:36.444701 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:36.444701 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:36.444779 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:36.445019 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:36.445020 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:36.445077 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 05:17:36.445150 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 05:17:36.445260 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 05:17:36.445837 4625 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.143.188" port: 41587 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:17:36.446214 4627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.446102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:41587 (local address 127.0.0.1:48142)
0811 05:17:36.446152 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:36.446191 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.188:41587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260811 05:17:36.446527 4627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.446437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:41587 (local address 127.0.0.1:48146)
0811 05:17:36.446482 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:36.446508 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.188:41587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260811 05:17:36.446699 4625 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.143.188:41587: Network error: Client connection negotiation failed: client connection to 127.28.143.188:41587: connect: Connection refused (error 111)
I20260811 05:17:36.452760 4773 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.143.188" port: 41587 } attempt: 1
I20260811 05:17:36.453430 4775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.453233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:41587 (local address 127.0.0.1:48162)
0811 05:17:36.453344 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:36.453395 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.188:41587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 05:17:36.453977 4775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.453776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:41587 (local address 127.0.0.1:48174)
0811 05:17:36.453914 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:36.453949 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.188:41587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
W20260811 05:17:36.454134 4773 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.143.188:41587: Network error: Client connection negotiation failed: client connection to 127.28.143.188:41587: connect: Connection refused (error 111)
I20260811 05:17:36.477986 4625 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.143.188" port: 41587 } attempt: 1
I20260811 05:17:36.478632 4627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.478455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:41587 (local address 127.0.0.1:48184)
0811 05:17:36.478548 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:36.478597 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.188:41587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 05:17:36.479017 4627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.478936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:41587 (local address 127.0.0.1:48196)
0811 05:17:36.478968 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:36.478997 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.188:41587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260811 05:17:36.479127 4625 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.143.188:41587: Network error: Client connection negotiation failed: client connection to 127.28.143.188:41587: connect: Connection refused (error 111)
W20260811 05:17:36.506701 4785 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 05:17:36.507002 4785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:36.507032 4785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:36.507059 4785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:36.508320 4785 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:36.508369 4785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:36.508388 4785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:36.508404 4785 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:36.508419 4785 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:36.510097 4785 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:45007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:38881,127.28.143.189:44537,127.28.143.188:41587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.188.keytab
--principal=kudu/127.28.143.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.188:41587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.28.143.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4785
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:17:36.510397 4785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:36.510644 4785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:36.515647 4797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:36.515698 4794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:36.515738 4785 server_base.cc:1034] running on GCE node
W20260811 05:17:36.515686 4802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:36.516216 4785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:36.516547 4785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:36.517748 4785 hybrid_clock.cc:648] HybridClock initialized: now 1786425456517732 us; error 27 us; skew 500 ppm
Aug 11 05:17:36 dist-test-slave-f7th krb5kdc[4462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425456, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:36.520658 4785 init.cc:377] Logged in from keytab as kudu/127.28.143.188@KRBTEST.COM (short username kudu)
I20260811 05:17:36.521162 4785 webserver.cc:460] Webserver started at http://127.28.143.188:38599/ using document root <none> and password file <none>
I20260811 05:17:36.521396 4785 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:36.521443 4785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:36.521574 4785 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:36.522320 4773 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.143.188" port: 41587 } attempt: 2
I20260811 05:17:36.522781 4775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.522628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:41587 (local address 127.0.0.1:48200)
0811 05:17:36.522698 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:36.522741 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.188:41587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 05:17:36.522763 4785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-2/data/instance:
uuid: "bae8e07eb8a945938c26b9e95cd9d6fe"
format_stamp: "Formatted at 2026-08-11 05:17:36 on dist-test-slave-f7th"
server_key: "c476bb02698fa13ad87e5448bdfe7f7a"
server_key_iv: "865cc2857ac317bab72f3d0da80a8ddb"
server_key_version: "encryptionkey@0"
I20260811 05:17:36.523180 4775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.523084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:41587 (local address 127.0.0.1:48212)
0811 05:17:36.523120 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:36.523152 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.188:41587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260811 05:17:36.523193 4785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-2/wal/instance:
uuid: "bae8e07eb8a945938c26b9e95cd9d6fe"
format_stamp: "Formatted at 2026-08-11 05:17:36 on dist-test-slave-f7th"
server_key: "c476bb02698fa13ad87e5448bdfe7f7a"
server_key_iv: "865cc2857ac317bab72f3d0da80a8ddb"
server_key_version: "encryptionkey@0"
W20260811 05:17:36.523326 4773 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.143.188:41587: Network error: Client connection negotiation failed: client connection to 127.28.143.188:41587: connect: Connection refused (error 111)
I20260811 05:17:36.524667 4785 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 05:17:36.525553 4812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:36.525801 4785 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 05:17:36.525970 4785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-2/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-2/wal
uuid: "bae8e07eb8a945938c26b9e95cd9d6fe"
format_stamp: "Formatted at 2026-08-11 05:17:36 on dist-test-slave-f7th"
server_key: "c476bb02698fa13ad87e5448bdfe7f7a"
server_key_iv: "865cc2857ac317bab72f3d0da80a8ddb"
server_key_version: "encryptionkey@0"
I20260811 05:17:36.526053 4785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:36.554298 4625 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.143.188" port: 41587 } attempt: 2
I20260811 05:17:36.554869 4627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.554705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:41587 (local address 127.0.0.1:48220)
0811 05:17:36.554787 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:36.554836 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.188:41587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 05:17:36.555241 4627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.555155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:41587 (local address 127.0.0.1:48226)
0811 05:17:36.555193 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:36.555222 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.188:41587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260811 05:17:36.555323 4625 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.143.188:41587: Network error: Client connection negotiation failed: client connection to 127.28.143.188:41587: connect: Connection refused (error 111)
I20260811 05:17:36.558863 4785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:36.561012 4785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:36.561141 4785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:36.565027 4785 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.188:41587
I20260811 05:17:36.565063 4899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.188:41587 every 8 connection(s)
I20260811 05:17:36.565495 4785 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-2/data/info.pb
I20260811 05:17:36.566517 4903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:36.568351 4903 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.143.190" port: 38881 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:17:36.570696 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 4785
I20260811 05:17:36.570814 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425159060450-29246-0/minicluster-data/master-2/wal/instance
I20260811 05:17:36.570999 29246 external_mini_cluster.cc:1428] Setting key ee5c912843a58b10f2547e6297d45550
Aug 11 05:17:36 dist-test-slave-f7th krb5kdc[4462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425456, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.188@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
Aug 11 05:17:36 dist-test-slave-f7th krb5kdc[4462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.143.188@KRBTEST.COM
I20260811 05:17:36.578802 4632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.569250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55460 (local address 127.28.143.190:38881)
0811 05:17:36.569322 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:36.569324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:36.569539 (+ 215us) server_negotiation.cc:407] Connection header received
0811 05:17:36.570028 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:36.570029 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:36.570039 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:36.570104 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:36.570898 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.571580 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.572968 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.573280 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.573358 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:36.575772 (+ 2414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:36.575773 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:36.575774 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:36.575781 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:36.577170 (+ 1389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:36.577655 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:36.577656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:36.577656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:36.577713 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:36.578000 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:36.578001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:36.578002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:36.578194 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:36.578264 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:36.578618 (+ 354us) server_negotiation.cc:299] Negotiation successful
0811 05:17:36.578705 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 05:17:36.578928 4906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.569200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:38881 (local address 127.0.0.1:55460)
0811 05:17:36.569406 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:36.569452 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:36.569941 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:36.570162 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:36.570165 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:36.570459 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:36.570774 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:36.570782 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.571667 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.571668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:36.572883 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:36.572886 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.573356 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.573356 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:36.573458 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:36.573969 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:17:36.573981 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:36.575672 (+ 1691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:36.577250 (+ 1578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:36.577259 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:36.577263 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:36.577589 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:36.577777 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:36.577778 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:36.577778 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:36.577923 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:36.578268 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:36.578274 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:36.578440 (+ 166us) client_negotiation.cc:769] Sending connection context
0811 05:17:36.578577 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 05:17:36.578732 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":106,"threads_started":1}
I20260811 05:17:36.579530 4903 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.143.189" port: 44537 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:17:36.580535 4908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.571418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48238 (local address 127.28.143.188:41587)
0811 05:17:36.571709 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:36.571713 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:36.571742 (+ 29us) server_negotiation.cc:407] Connection header received
0811 05:17:36.571761 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:36.571763 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:36.571780 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:36.571830 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:36.572604 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.573547 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.574509 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.574841 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.574897 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:36.577524 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:36.577532 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:36.577533 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:36.577539 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:36.579320 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:36.579720 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:36.579723 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:36.579724 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:36.579780 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:36.579961 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:36.579962 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:36.579962 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:36.580188 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:36.580349 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:36.580402 (+ 53us) server_negotiation.cc:299] Negotiation successful
0811 05:17:36.580460 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":259,"thread_start_us":99,"threads_started":1}
Aug 11 05:17:36 dist-test-slave-f7th krb5kdc[4462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425456, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.188@KRBTEST.COM for kudu/127.28.143.189@KRBTEST.COM
Aug 11 05:17:36 dist-test-slave-f7th krb5kdc[4462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:17:36.586894 4777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.579869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58474 (local address 127.28.143.189:44537)
0811 05:17:36.579937 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:36.579939 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:36.579948 (+ 9us) server_negotiation.cc:407] Connection header received
0811 05:17:36.579972 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:36.579974 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:36.579984 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:36.580040 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:36.580297 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.580945 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.581863 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.582085 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.582142 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:36.584006 (+ 1864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:36.584008 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:36.584009 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:36.584018 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:36.585560 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:36.586000 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:36.586001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:36.586002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:36.586059 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:36.586265 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:36.586266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:36.586266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:36.586414 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:36.586491 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:36.586693 (+ 202us) server_negotiation.cc:299] Negotiation successful
0811 05:17:36.586777 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 05:17:36.586869 4906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.579804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.189:44537 (local address 127.0.0.1:58474)
0811 05:17:36.579851 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:36.579860 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:36.579894 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:36.580056 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:36.580057 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:36.580196 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:36.580235 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:36.580237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.581011 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.581012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:36.581791 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:36.581794 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.582144 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.582145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:36.582200 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:36.582588 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:17:36.582592 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:36.583865 (+ 1273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:36.585697 (+ 1832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:36.585698 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:36.585699 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:36.585910 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:36.586125 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:36.586126 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:36.586127 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:36.586197 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:36.586517 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:36.586518 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:36.586575 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 05:17:36.586662 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 05:17:36.586782 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260811 05:17:36.587332 4903 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.143.188" port: 41587 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:17:36.589057 4632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.581411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55466 (local address 127.28.143.190:38881)
0811 05:17:36.581457 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:36.581459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:36.581501 (+ 42us) server_negotiation.cc:407] Connection header received
0811 05:17:36.581568 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:36.581569 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:36.581578 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:36.581618 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:36.582095 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.582647 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.583461 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.583674 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.583741 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:36.586259 (+ 2518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:36.586260 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:36.586261 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:36.586268 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:36.587861 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:36.588212 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:36.588214 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:36.588214 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:36.588305 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:36.588511 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:36.588512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:36.588512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:36.588664 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:36.588869 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:36.588920 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 05:17:36.588992 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
Aug 11 05:17:36 dist-test-slave-f7th krb5kdc[4462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425456, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.188@KRBTEST.COM for kudu/127.28.143.188@KRBTEST.COM
Aug 11 05:17:36 dist-test-slave-f7th krb5kdc[4462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.143.189@KRBTEST.COM
I20260811 05:17:36.594496 4906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.587647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:41587 (local address 127.0.0.1:48252)
0811 05:17:36.587707 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:36.587716 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:36.587752 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:36.587852 (+ 100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:36.587853 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:36.588009 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:36.588050 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:36.588052 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.588901 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.588902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:36.589557 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:36.589560 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.590009 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.590010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:36.590053 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:36.590405 (+ 352us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:17:36.590409 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:36.591717 (+ 1308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:36.593415 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:36.593416 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:36.593416 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:36.593613 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:36.593831 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:36.593832 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:36.593832 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:36.593885 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:36.594167 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:36.594168 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:36.594206 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 05:17:36.594259 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 05:17:36.594344 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 05:17:36.594504 4908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.587698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48252 (local address 127.28.143.188:41587)
0811 05:17:36.587751 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:36.587753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:36.587760 (+ 7us) server_negotiation.cc:407] Connection header received
0811 05:17:36.587798 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:36.587799 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:36.587807 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:36.587852 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:36.588116 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.588816 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.589631 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.589895 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.589963 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:36.591805 (+ 1842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:36.591806 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:36.591807 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:36.591814 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:36.593319 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:36.593692 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:36.593693 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:36.593693 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:36.593768 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:36.593965 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:36.593966 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:36.593966 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:36.594100 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:36.594170 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:36.594299 (+ 129us) server_negotiation.cc:299] Negotiation successful
0811 05:17:36.594401 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":35}
I20260811 05:17:36.596421 4903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bae8e07eb8a945938c26b9e95cd9d6fe: Bootstrap starting.
I20260811 05:17:36.597596 4903 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bae8e07eb8a945938c26b9e95cd9d6fe: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:36.597688 4777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.589937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58482 (local address 127.28.143.189:44537)
0811 05:17:36.589997 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:36.589998 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:36.590008 (+ 10us) server_negotiation.cc:407] Connection header received
0811 05:17:36.590032 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:36.590033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:36.590043 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:36.590088 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:36.590573 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.591168 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.592052 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.592312 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.592428 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:36.594898 (+ 2470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:36.594899 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:36.594899 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:36.594908 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:36.596420 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:36.596834 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:36.596835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:36.596836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:36.596907 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:36.597141 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:36.597142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:36.597142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:36.597281 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:36.597487 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:36.597529 (+ 42us) server_negotiation.cc:299] Negotiation successful
0811 05:17:36.597606 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 05:17:36.597949 4903 log.cc:824] T 00000000000000000000000000000000 P bae8e07eb8a945938c26b9e95cd9d6fe: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:36.598906 4903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bae8e07eb8a945938c26b9e95cd9d6fe: No bootstrap required, opened a new log
I20260811 05:17:36.599558 29246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:17:36.600471 4903 raft_consensus.cc:348] T 00000000000000000000000000000000 P bae8e07eb8a945938c26b9e95cd9d6fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc6daa2d3811400eb1cc9a35bd938de6" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 38881 } } peers { permanent_uuid: "6ff0f0c96b694fa696b89c27e396b31c" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 44537 } } peers { permanent_uuid: "bae8e07eb8a945938c26b9e95cd9d6fe" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 41587 } }
I20260811 05:17:36.600687 4903 raft_consensus.cc:729] T 00000000000000000000000000000000 P bae8e07eb8a945938c26b9e95cd9d6fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bae8e07eb8a945938c26b9e95cd9d6fe, State: Initialized, Role: FOLLOWER
I20260811 05:17:36.600788 4903 consensus_queue.cc:260] T 00000000000000000000000000000000 P bae8e07eb8a945938c26b9e95cd9d6fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc6daa2d3811400eb1cc9a35bd938de6" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 38881 } } peers { permanent_uuid: "6ff0f0c96b694fa696b89c27e396b31c" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 44537 } } peers { permanent_uuid: "bae8e07eb8a945938c26b9e95cd9d6fe" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 41587 } }
I20260811 05:17:36.601159 4903 sys_catalog.cc:564] T 00000000000000000000000000000000 P bae8e07eb8a945938c26b9e95cd9d6fe [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:36.601135 4914 sys_catalog.cc:455] T 00000000000000000000000000000000 P bae8e07eb8a945938c26b9e95cd9d6fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc6daa2d3811400eb1cc9a35bd938de6" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 38881 } } peers { permanent_uuid: "6ff0f0c96b694fa696b89c27e396b31c" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 44537 } } peers { permanent_uuid: "bae8e07eb8a945938c26b9e95cd9d6fe" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 41587 } } }
I20260811 05:17:36.601222 4914 sys_catalog.cc:458] T 00000000000000000000000000000000 P bae8e07eb8a945938c26b9e95cd9d6fe [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:17:36.600729 4578 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "cc6daa2d3811400eb1cc9a35bd938de6"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55466
W20260811 05:17:36.603150 4933 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bae8e07eb8a945938c26b9e95cd9d6fe: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:17:36.603217 4933 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:17:36.603457 4785 master_runner.cc:186] Error getting master registration for 127.28.143.190:38881: OK, instance_id { permanent_uuid: "cc6daa2d3811400eb1cc9a35bd938de6" instance_seqno: 1786425456209253 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 05:17:36.605681 4785 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 05:17:36.614500 4773 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.143.188" port: 41587 } attempt: 3
Aug 11 05:17:36 dist-test-slave-f7th krb5kdc[4462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425456, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.189@KRBTEST.COM for kudu/127.28.143.188@KRBTEST.COM
I20260811 05:17:36.623299 4908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.615085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48256 (local address 127.28.143.188:41587)
0811 05:17:36.615150 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:36.615152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:36.615164 (+ 12us) server_negotiation.cc:407] Connection header received
0811 05:17:36.615191 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:36.615192 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:36.615203 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:36.615249 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:36.615627 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.616099 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.617148 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.617339 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.617443 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:36.619497 (+ 2054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:36.619499 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:36.619499 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:36.619508 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:36.621759 (+ 2251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:36.622340 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:36.622342 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:36.622343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:36.622422 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:36.622649 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:36.622650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:36.622650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:36.622870 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:36.622939 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:36.623101 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 05:17:36.623195 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 05:17:36.623306 4775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.614984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:41587 (local address 127.0.0.1:48256)
0811 05:17:36.615059 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:36.615072 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:36.615124 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:36.615262 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:36.615263 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:36.615483 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:36.615541 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:36.615543 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.616189 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.616191 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:36.617032 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:36.617037 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.617408 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.617410 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:36.617462 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:36.617992 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:17:36.617997 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:36.619352 (+ 1355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:36.621894 (+ 2542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:36.621896 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:36.621897 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:36.622172 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:36.622509 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:36.622510 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:36.622511 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:36.622587 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:36.622937 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:36.622938 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:36.622997 (+ 59us) client_negotiation.cc:769] Sending connection context
0811 05:17:36.623070 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 05:17:36.623193 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 05:17:36.625296 4773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ff0f0c96b694fa696b89c27e396b31c: Bootstrap starting.
I20260811 05:17:36.626152 4773 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ff0f0c96b694fa696b89c27e396b31c: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:36.626482 4773 log.cc:824] T 00000000000000000000000000000000 P 6ff0f0c96b694fa696b89c27e396b31c: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:36.627480 4773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ff0f0c96b694fa696b89c27e396b31c: No bootstrap required, opened a new log
I20260811 05:17:36.629731 4773 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ff0f0c96b694fa696b89c27e396b31c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc6daa2d3811400eb1cc9a35bd938de6" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 38881 } } peers { permanent_uuid: "6ff0f0c96b694fa696b89c27e396b31c" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 44537 } } peers { permanent_uuid: "bae8e07eb8a945938c26b9e95cd9d6fe" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 41587 } }
I20260811 05:17:36.629911 4773 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ff0f0c96b694fa696b89c27e396b31c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ff0f0c96b694fa696b89c27e396b31c, State: Initialized, Role: FOLLOWER
I20260811 05:17:36.630008 4773 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ff0f0c96b694fa696b89c27e396b31c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc6daa2d3811400eb1cc9a35bd938de6" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 38881 } } peers { permanent_uuid: "6ff0f0c96b694fa696b89c27e396b31c" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 44537 } } peers { permanent_uuid: "bae8e07eb8a945938c26b9e95cd9d6fe" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 41587 } }
I20260811 05:17:36.630363 4936 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ff0f0c96b694fa696b89c27e396b31c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc6daa2d3811400eb1cc9a35bd938de6" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 38881 } } peers { permanent_uuid: "6ff0f0c96b694fa696b89c27e396b31c" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 44537 } } peers { permanent_uuid: "bae8e07eb8a945938c26b9e95cd9d6fe" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 41587 } } }
I20260811 05:17:36.630431 4773 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ff0f0c96b694fa696b89c27e396b31c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:36.630486 4936 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ff0f0c96b694fa696b89c27e396b31c [sys.catalog]: This master's current role is: FOLLOWER
W20260811 05:17:36.632581 4961 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6ff0f0c96b694fa696b89c27e396b31c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:17:36.632692 4961 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:17:36.632793 4640 master_runner.cc:186] Error getting master registration for 127.28.143.190:38881: OK, instance_id { permanent_uuid: "cc6daa2d3811400eb1cc9a35bd938de6" instance_seqno: 1786425456209253 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 05:17:36.635660 4640 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 05:17:36.654454 4625 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.143.188" port: 41587 } attempt: 3
Aug 11 05:17:36 dist-test-slave-f7th krb5kdc[4462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425456, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for kudu/127.28.143.188@KRBTEST.COM
I20260811 05:17:36.662472 4627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.654808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:41587 (local address 127.0.0.1:48262)
0811 05:17:36.654893 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:36.654905 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:36.654959 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:36.655199 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:36.655200 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:36.655419 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:36.655475 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:36.655477 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.656377 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.656378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:36.656960 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:36.656963 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.657431 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.657432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:36.657483 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:36.657992 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:17:36.657997 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:36.659595 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:36.661442 (+ 1847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:36.661443 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:36.661444 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:36.661621 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:36.661801 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:36.661817 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:36.661817 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:36.661877 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:36.662128 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:36.662129 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:36.662192 (+ 63us) client_negotiation.cc:769] Sending connection context
0811 05:17:36.662250 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 05:17:36.662362 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 05:17:36.662472 4908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.655034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48262 (local address 127.28.143.188:41587)
0811 05:17:36.655072 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:36.655074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:36.655082 (+ 8us) server_negotiation.cc:407] Connection header received
0811 05:17:36.655112 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:36.655113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:36.655123 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:36.655171 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:36.655538 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.656293 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.657054 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.657356 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.657422 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:36.659704 (+ 2282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:36.659706 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:36.659707 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:36.659716 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:36.661359 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:36.661690 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:36.661691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:36.661692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:36.661752 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:36.661923 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:36.661924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:36.661925 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:36.662079 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:36.662138 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:36.662286 (+ 148us) server_negotiation.cc:299] Negotiation successful
0811 05:17:36.662362 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260811 05:17:36.664587 4625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc6daa2d3811400eb1cc9a35bd938de6: Bootstrap starting.
I20260811 05:17:36.665421 4625 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc6daa2d3811400eb1cc9a35bd938de6: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:36.665750 4625 log.cc:824] T 00000000000000000000000000000000 P cc6daa2d3811400eb1cc9a35bd938de6: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:36.666636 4625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc6daa2d3811400eb1cc9a35bd938de6: No bootstrap required, opened a new log
I20260811 05:17:36.668048 4625 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc6daa2d3811400eb1cc9a35bd938de6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc6daa2d3811400eb1cc9a35bd938de6" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 38881 } } peers { permanent_uuid: "6ff0f0c96b694fa696b89c27e396b31c" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 44537 } } peers { permanent_uuid: "bae8e07eb8a945938c26b9e95cd9d6fe" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 41587 } }
I20260811 05:17:36.668217 4625 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc6daa2d3811400eb1cc9a35bd938de6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc6daa2d3811400eb1cc9a35bd938de6, State: Initialized, Role: FOLLOWER
I20260811 05:17:36.668310 4625 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc6daa2d3811400eb1cc9a35bd938de6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc6daa2d3811400eb1cc9a35bd938de6" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 38881 } } peers { permanent_uuid: "6ff0f0c96b694fa696b89c27e396b31c" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 44537 } } peers { permanent_uuid: "bae8e07eb8a945938c26b9e95cd9d6fe" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 41587 } }
I20260811 05:17:36.668725 4625 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc6daa2d3811400eb1cc9a35bd938de6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:36.668777 4578 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc6daa2d3811400eb1cc9a35bd938de6 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 05:17:36.668821 4578 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc6daa2d3811400eb1cc9a35bd938de6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:36.669657 4578 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc6daa2d3811400eb1cc9a35bd938de6 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc6daa2d3811400eb1cc9a35bd938de6" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 38881 } } peers { permanent_uuid: "6ff0f0c96b694fa696b89c27e396b31c" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 44537 } } peers { permanent_uuid: "bae8e07eb8a945938c26b9e95cd9d6fe" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 41587 } }
I20260811 05:17:36.669960 4578 leader_election.cc:288] T 00000000000000000000000000000000 P cc6daa2d3811400eb1cc9a35bd938de6 [CANDIDATE]: Term 1 election: Requested vote from peers 6ff0f0c96b694fa696b89c27e396b31c (127.28.143.189:44537), bae8e07eb8a945938c26b9e95cd9d6fe (127.28.143.188:41587)
W20260811 05:17:36.670722 4996 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cc6daa2d3811400eb1cc9a35bd938de6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:17:36.670783 4996 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:17:36.670769 4862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cc6daa2d3811400eb1cc9a35bd938de6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "bae8e07eb8a945938c26b9e95cd9d6fe"
I20260811 05:17:36.670892 4862 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bae8e07eb8a945938c26b9e95cd9d6fe [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:36.670914 4727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cc6daa2d3811400eb1cc9a35bd938de6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6ff0f0c96b694fa696b89c27e396b31c"
I20260811 05:17:36.671092 4727 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ff0f0c96b694fa696b89c27e396b31c [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:36.671757 4862 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bae8e07eb8a945938c26b9e95cd9d6fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc6daa2d3811400eb1cc9a35bd938de6 in term 1.
I20260811 05:17:36.672049 4727 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6ff0f0c96b694fa696b89c27e396b31c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc6daa2d3811400eb1cc9a35bd938de6 in term 1.
I20260811 05:17:36.672434 4519 leader_election.cc:302] T 00000000000000000000000000000000 P cc6daa2d3811400eb1cc9a35bd938de6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bae8e07eb8a945938c26b9e95cd9d6fe, cc6daa2d3811400eb1cc9a35bd938de6; no voters:
I20260811 05:17:36.668660 4966 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc6daa2d3811400eb1cc9a35bd938de6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc6daa2d3811400eb1cc9a35bd938de6" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 38881 } } peers { permanent_uuid: "6ff0f0c96b694fa696b89c27e396b31c" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 44537 } } peers { permanent_uuid: "bae8e07eb8a945938c26b9e95cd9d6fe" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 41587 } } }
I20260811 05:17:36.673375 4966 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc6daa2d3811400eb1cc9a35bd938de6 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:17:36.673493 4966 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc6daa2d3811400eb1cc9a35bd938de6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:36.673600 4966 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc6daa2d3811400eb1cc9a35bd938de6 [term 1 LEADER]: Becoming Leader. State: Replica: cc6daa2d3811400eb1cc9a35bd938de6, State: Running, Role: LEADER
I20260811 05:17:36.673734 4966 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc6daa2d3811400eb1cc9a35bd938de6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc6daa2d3811400eb1cc9a35bd938de6" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 38881 } } peers { permanent_uuid: "6ff0f0c96b694fa696b89c27e396b31c" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 44537 } } peers { permanent_uuid: "bae8e07eb8a945938c26b9e95cd9d6fe" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 41587 } }
I20260811 05:17:36.674295 4966 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc6daa2d3811400eb1cc9a35bd938de6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc6daa2d3811400eb1cc9a35bd938de6. Latest consensus state: current_term: 1 leader_uuid: "cc6daa2d3811400eb1cc9a35bd938de6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc6daa2d3811400eb1cc9a35bd938de6" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 38881 } } peers { permanent_uuid: "6ff0f0c96b694fa696b89c27e396b31c" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 44537 } } peers { permanent_uuid: "bae8e07eb8a945938c26b9e95cd9d6fe" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 41587 } } }
I20260811 05:17:36.674428 4966 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc6daa2d3811400eb1cc9a35bd938de6 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:36.674961 4487 master_runner.cc:263] No leader master found from master cc6daa2d3811400eb1cc9a35bd938de6
I20260811 05:17:36.675014 4487 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 05:17:36.675365 5012 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:17:36.675632 5012 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:17:36.683058 4908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.674840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48278 (local address 127.28.143.188:41587)
0811 05:17:36.674882 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:36.674884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:36.675350 (+ 466us) server_negotiation.cc:407] Connection header received
0811 05:17:36.675453 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:36.675454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:36.675465 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:36.675494 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:36.676308 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.677049 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.678062 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.678353 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.678394 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:36.679996 (+ 1602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:36.679998 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:36.679998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:36.680005 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:36.681664 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:36.682023 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:36.682024 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:36.682025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:36.682096 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:36.682297 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:36.682298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:36.682298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:36.682429 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:36.682499 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:36.682851 (+ 352us) server_negotiation.cc:299] Negotiation successful
0811 05:17:36.682957 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 05:17:36.686447 4862 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bae8e07eb8a945938c26b9e95cd9d6fe [term 1 FOLLOWER]: Refusing update from remote peer cc6daa2d3811400eb1cc9a35bd938de6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:17:36.686987 4966 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cc6daa2d3811400eb1cc9a35bd938de6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bae8e07eb8a945938c26b9e95cd9d6fe" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 41587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:17:36.688743 4914 sys_catalog.cc:455] T 00000000000000000000000000000000 P bae8e07eb8a945938c26b9e95cd9d6fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc6daa2d3811400eb1cc9a35bd938de6. Latest consensus state: current_term: 1 leader_uuid: "cc6daa2d3811400eb1cc9a35bd938de6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc6daa2d3811400eb1cc9a35bd938de6" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 38881 } } peers { permanent_uuid: "6ff0f0c96b694fa696b89c27e396b31c" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 44537 } } peers { permanent_uuid: "bae8e07eb8a945938c26b9e95cd9d6fe" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 41587 } } }
I20260811 05:17:36.688853 4914 sys_catalog.cc:458] T 00000000000000000000000000000000 P bae8e07eb8a945938c26b9e95cd9d6fe [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:17:36.689533 4727 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6ff0f0c96b694fa696b89c27e396b31c [term 1 FOLLOWER]: Refusing update from remote peer cc6daa2d3811400eb1cc9a35bd938de6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:17:36.689910 4966 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cc6daa2d3811400eb1cc9a35bd938de6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ff0f0c96b694fa696b89c27e396b31c" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 44537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:17:36.690806 4632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.676886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55468 (local address 127.28.143.190:38881)
0811 05:17:36.676927 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:36.676929 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:36.683478 (+ 6549us) server_negotiation.cc:407] Connection header received
0811 05:17:36.683596 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:36.683597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:36.683610 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:36.683642 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:36.684241 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.685020 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.685956 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.686236 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.686280 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:36.687681 (+ 1401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:36.687683 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:36.687683 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:36.687691 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:36.689421 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:36.689864 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:36.689866 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:36.689866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:36.689929 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:36.690151 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:36.690152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:36.690152 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:36.690299 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:36.690386 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:36.690632 (+ 246us) server_negotiation.cc:299] Negotiation successful
0811 05:17:36.690721 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 05:17:36.691493 4936 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ff0f0c96b694fa696b89c27e396b31c [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc6daa2d3811400eb1cc9a35bd938de6. Latest consensus state: current_term: 1 leader_uuid: "cc6daa2d3811400eb1cc9a35bd938de6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc6daa2d3811400eb1cc9a35bd938de6" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 38881 } } peers { permanent_uuid: "6ff0f0c96b694fa696b89c27e396b31c" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 44537 } } peers { permanent_uuid: "bae8e07eb8a945938c26b9e95cd9d6fe" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 41587 } } }
I20260811 05:17:36.691588 4936 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ff0f0c96b694fa696b89c27e396b31c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:17:36.692934 4966 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc6daa2d3811400eb1cc9a35bd938de6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cc6daa2d3811400eb1cc9a35bd938de6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc6daa2d3811400eb1cc9a35bd938de6" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 38881 } } peers { permanent_uuid: "6ff0f0c96b694fa696b89c27e396b31c" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 44537 } } peers { permanent_uuid: "bae8e07eb8a945938c26b9e95cd9d6fe" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 41587 } } }
I20260811 05:17:36.693034 4966 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc6daa2d3811400eb1cc9a35bd938de6 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:36.693270 4777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.675156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58494 (local address 127.28.143.189:44537)
0811 05:17:36.675200 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:36.675202 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:36.677121 (+ 1919us) server_negotiation.cc:407] Connection header received
0811 05:17:36.677223 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:36.677224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:36.677235 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:36.677262 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:36.677977 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.678635 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.679582 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.679870 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.679926 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:36.682456 (+ 2530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:36.682458 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:36.682458 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:36.682467 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:36.684769 (+ 2302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:36.692048 (+ 7279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:36.692051 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:36.692051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:36.692134 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:36.692378 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:36.692379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:36.692379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:36.692526 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:36.692633 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:36.692985 (+ 352us) server_negotiation.cc:299] Negotiation successful
0811 05:17:36.693181 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 05:17:36.693574 5025 mvcc.cc:205] Tried to move back new op lower bound from 7317198670569635840 to 7317198670536634368. Current Snapshot: MvccSnapshot[applied={T|T < 7317198670569635840}]
I20260811 05:17:36.693712 4936 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ff0f0c96b694fa696b89c27e396b31c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cc6daa2d3811400eb1cc9a35bd938de6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc6daa2d3811400eb1cc9a35bd938de6" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 38881 } } peers { permanent_uuid: "6ff0f0c96b694fa696b89c27e396b31c" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 44537 } } peers { permanent_uuid: "bae8e07eb8a945938c26b9e95cd9d6fe" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 41587 } } }
I20260811 05:17:36.693794 4936 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ff0f0c96b694fa696b89c27e396b31c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:17:36.695153 4966 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc6daa2d3811400eb1cc9a35bd938de6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cc6daa2d3811400eb1cc9a35bd938de6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc6daa2d3811400eb1cc9a35bd938de6" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 38881 } } peers { permanent_uuid: "6ff0f0c96b694fa696b89c27e396b31c" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 44537 } } peers { permanent_uuid: "bae8e07eb8a945938c26b9e95cd9d6fe" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 41587 } } }
I20260811 05:17:36.695255 4966 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc6daa2d3811400eb1cc9a35bd938de6 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:36.695716 4914 sys_catalog.cc:455] T 00000000000000000000000000000000 P bae8e07eb8a945938c26b9e95cd9d6fe [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cc6daa2d3811400eb1cc9a35bd938de6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc6daa2d3811400eb1cc9a35bd938de6" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 38881 } } peers { permanent_uuid: "6ff0f0c96b694fa696b89c27e396b31c" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 44537 } } peers { permanent_uuid: "bae8e07eb8a945938c26b9e95cd9d6fe" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 41587 } } }
I20260811 05:17:36.695815 4914 sys_catalog.cc:458] T 00000000000000000000000000000000 P bae8e07eb8a945938c26b9e95cd9d6fe [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:17:36.701334 5012 catalog_manager.cc:1347] Generated new cluster ID: a3c4faa0e3e941628786da09f71aae99
I20260811 05:17:36.701464 5012 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:17:36.728585 5012 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:17:36.729319 5012 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:17:36.737304 5012 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc6daa2d3811400eb1cc9a35bd938de6: Generated new TSK 0
I20260811 05:17:36.737540 5012 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 05:17:36.766150 4777 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:17:36.764585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58506 (local address 127.28.143.189:44537)
0811 05:17:36.764659 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:36.764661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:36.765403 (+ 742us) server_negotiation.cc:407] Connection header received
0811 05:17:36.765522 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:36.765523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:36.765535 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:36.765564 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:36.766069 (+ 505us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58506: BlockingRecv error: recv got EOF from 127.0.0.1:58506 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260811 05:17:36.767062 4632 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:17:36.765057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55480 (local address 127.28.143.190:38881)
0811 05:17:36.765101 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:36.765102 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:36.766467 (+ 1365us) server_negotiation.cc:407] Connection header received
0811 05:17:36.766566 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:36.766567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:36.766579 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:36.766608 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:36.766978 (+ 370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55480: BlockingRecv error: recv got EOF from 127.0.0.1:55480 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260811 05:17:36.768133 4908 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:17:36.767463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48284 (local address 127.28.143.188:41587)
0811 05:17:36.767504 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:36.767505 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:36.767626 (+ 121us) server_negotiation.cc:407] Connection header received
0811 05:17:36.767721 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:36.767722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:36.767732 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:36.767761 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:36.768074 (+ 313us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48284: BlockingRecv error: recv got EOF from 127.0.0.1:48284 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260811 05:17:36.770753 4777 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:17:36.769893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58516 (local address 127.28.143.189:44537)
0811 05:17:36.769937 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:36.769939 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:36.770149 (+ 210us) server_negotiation.cc:407] Connection header received
0811 05:17:36.770256 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:36.770257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:36.770277 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:36.770305 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:36.770699 (+ 394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58516: BlockingRecv error: recv got EOF from 127.0.0.1:58516 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260811 05:17:36.774224 4777 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:17:36.773327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58528 (local address 127.28.143.189:44537)
0811 05:17:36.773377 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:36.773379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:36.773664 (+ 285us) server_negotiation.cc:407] Connection header received
0811 05:17:36.773765 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:36.773766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:36.773777 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:36.773806 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:36.774166 (+ 360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58528: BlockingRecv error: recv got EOF from 127.0.0.1:58528 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260811 05:17:36.778306 4777 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:17:36.777434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58542 (local address 127.28.143.189:44537)
0811 05:17:36.777475 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:36.777476 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:36.777745 (+ 269us) server_negotiation.cc:407] Connection header received
0811 05:17:36.777845 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:36.777846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:36.777860 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:36.777887 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:36.778247 (+ 360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58542: BlockingRecv error: recv got EOF from 127.0.0.1:58542 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260811 05:17:36.784557 4777 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:17:36.783601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58548 (local address 127.28.143.189:44537)
0811 05:17:36.783645 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:36.783647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:36.783946 (+ 299us) server_negotiation.cc:407] Connection header received
0811 05:17:36.784046 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:36.784047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:36.784061 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:36.784091 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:36.784500 (+ 409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58548: BlockingRecv error: recv got EOF from 127.0.0.1:58548 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260811 05:17:36.794872 4777 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:17:36.793869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58552 (local address 127.28.143.189:44537)
0811 05:17:36.793919 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:36.793922 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:36.794219 (+ 297us) server_negotiation.cc:407] Connection header received
0811 05:17:36.794327 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:36.794328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:36.794341 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:36.794385 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:36.794810 (+ 425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58552: BlockingRecv error: recv got EOF from 127.0.0.1:58552 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260811 05:17:36.813603 4777 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:17:36.812230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58558 (local address 127.28.143.189:44537)
0811 05:17:36.812275 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:36.812278 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:36.812577 (+ 299us) server_negotiation.cc:407] Connection header received
0811 05:17:36.812724 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:36.812725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:36.812738 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:36.812766 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:36.813538 (+ 772us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58558: BlockingRecv error: recv got EOF from 127.0.0.1:58558 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260811 05:17:36.847503 4777 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:17:36.846682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58574 (local address 127.28.143.189:44537)
0811 05:17:36.846767 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:36.846769 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:36.846783 (+ 14us) server_negotiation.cc:407] Connection header received
0811 05:17:36.846884 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:36.846885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:36.846900 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:36.846967 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:36.847433 (+ 466us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58574: BlockingRecv error: recv got EOF from 127.0.0.1:58574 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260811 05:17:36.913615 4777 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:17:36.912828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58586 (local address 127.28.143.189:44537)
0811 05:17:36.912883 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:36.912885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:36.912951 (+ 66us) server_negotiation.cc:407] Connection header received
0811 05:17:36.913006 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:36.913007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:36.913034 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:36.913098 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:36.913547 (+ 449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58586: BlockingRecv error: recv got EOF from 127.0.0.1:58586 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260811 05:17:37.043485 4777 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:17:37.042800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58588 (local address 127.28.143.189:44537)
0811 05:17:37.042867 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:37.042869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:37.042940 (+ 71us) server_negotiation.cc:407] Connection header received
0811 05:17:37.043027 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:37.043028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:37.043041 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:37.043091 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:37.043425 (+ 334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58588: BlockingRecv error: recv got EOF from 127.0.0.1:58588 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260811 05:17:37.301501 4777 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:17:37.300784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58594 (local address 127.28.143.189:44537)
0811 05:17:37.300840 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:37.300842 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:37.300904 (+ 62us) server_negotiation.cc:407] Connection header received
0811 05:17:37.300965 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:37.300966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:37.300977 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:37.301088 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:37.301441 (+ 353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58594: BlockingRecv error: recv got EOF from 127.0.0.1:58594 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 05:17:37.603736 4933 catalog_manager.cc:1259] Loaded cluster ID: a3c4faa0e3e941628786da09f71aae99
I20260811 05:17:37.603824 4933 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bae8e07eb8a945938c26b9e95cd9d6fe: loading cluster ID for follower catalog manager: success
I20260811 05:17:37.604714 4933 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bae8e07eb8a945938c26b9e95cd9d6fe: acquiring CA information for follower catalog manager: success
I20260811 05:17:37.605041 4933 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bae8e07eb8a945938c26b9e95cd9d6fe: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 05:17:37.633201 4961 catalog_manager.cc:1259] Loaded cluster ID: a3c4faa0e3e941628786da09f71aae99
I20260811 05:17:37.633283 4961 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6ff0f0c96b694fa696b89c27e396b31c: loading cluster ID for follower catalog manager: success
I20260811 05:17:37.634109 4961 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6ff0f0c96b694fa696b89c27e396b31c: acquiring CA information for follower catalog manager: success
I20260811 05:17:37.634449 4961 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6ff0f0c96b694fa696b89c27e396b31c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 05:17:37.817436 5146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:37.814786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58598 (local address 127.28.143.189:44537)
0811 05:17:37.814902 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:37.814904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:37.814915 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:17:37.814982 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:37.814983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:37.815035 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:37.815108 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:17:37.815235 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:37.815729 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:37.816515 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:37.816748 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:37.816813 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:37.817067 (+ 254us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:17:37.817188 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:17:37.817266 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:37.817338 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 05:17:37.817360 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":57,"threads_started":1}
I20260811 05:17:37.821822 5154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:37.819203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48300 (local address 127.28.143.188:41587)
0811 05:17:37.819322 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:37.819323 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:37.819332 (+ 9us) server_negotiation.cc:407] Connection header received
0811 05:17:37.819376 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:37.819376 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:37.819413 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:37.819484 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:17:37.819703 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:37.820203 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:37.820982 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:37.821200 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:37.821262 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:37.821378 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:17:37.821522 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:17:37.821583 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:37.821698 (+ 115us) server_negotiation.cc:299] Negotiation successful
0811 05:17:37.821725 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":60,"threads_started":1}
I20260811 05:17:37.826771 5154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:37.823708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48302 (local address 127.28.143.188:41587)
0811 05:17:37.823753 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:37.823755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:37.823817 (+ 62us) server_negotiation.cc:407] Connection header received
0811 05:17:37.823938 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:37.823939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:37.823951 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:37.824007 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:17:37.824305 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:37.825100 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:37.825901 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:37.826189 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:37.826248 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:37.826436 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:17:37.826469 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:17:37.826509 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:37.826664 (+ 155us) server_negotiation.cc:299] Negotiation successful
0811 05:17:37.826688 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 05:17:37.826941 5169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:37.823473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55488 (local address 127.28.143.190:38881)
0811 05:17:37.823677 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:37.823679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:37.823690 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:17:37.823819 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:37.823821 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:37.823880 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:37.824007 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:17:37.824210 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:37.824854 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:37.825876 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:37.826127 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:37.826193 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:37.826400 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:17:37.826527 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:17:37.826804 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:37.826822 (+ 18us) server_negotiation.cc:299] Negotiation successful
0811 05:17:37.826864 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":90,"threads_started":1}
I20260811 05:17:37.827481 5146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:37.823459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58604 (local address 127.28.143.189:44537)
0811 05:17:37.823502 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:37.823504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:37.823853 (+ 349us) server_negotiation.cc:407] Connection header received
0811 05:17:37.823962 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:37.823964 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:37.824103 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:37.824172 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:17:37.824379 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:37.825162 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:37.826146 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:37.826431 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:37.826492 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:37.826692 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:17:37.826727 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:17:37.826755 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:37.827375 (+ 620us) server_negotiation.cc:299] Negotiation successful
0811 05:17:37.827407 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 05:17:37.829175 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 4487
I20260811 05:17:37.847119 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 4640
I20260811 05:17:37.870282 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 4785
2026-08-11T05:17:37Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3158 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:17:37 dist-test-slave-f7th krb5kdc[5176](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:17:37 dist-test-slave-f7th krb5kdc[5176](info): set up 2 sockets
Aug 11 05:17:37 dist-test-slave-f7th krb5kdc[5176](info): commencing operation
krb5kdc: starting...
W20260811 05:17:39.090831 29246 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/krb5kdc/test-user.keytab.
Aug 11 05:17:39 dist-test-slave-f7th krb5kdc[5176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:17:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:17:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
Entry for principal kudu/127.28.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab.
I20260811 05:17:39.221648 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:33785
--webserver_interface=127.28.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:38901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:33785,127.28.143.189:35489,127.28.143.188:40551
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:39.307590 5199 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 05:17:39.307818 5199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:39.307843 5199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:39.307879 5199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:39.309072 5199 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:39.309114 5199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:39.309132 5199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:39.309149 5199 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:39.309163 5199 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:39.310680 5199 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:38901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:33785,127.28.143.189:35489,127.28.143.188:40551
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.190.keytab
--principal=kudu/127.28.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.190:33785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:17:39.310999 5199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:39.311228 5199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:39.316035 5209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:39.316012 5213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:39.316017 5210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:39.316303 5199 server_base.cc:1034] running on GCE node
I20260811 05:17:39.316743 5199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:39.317128 5199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:39.318293 5199 hybrid_clock.cc:648] HybridClock initialized: now 1786425459318273 us; error 37 us; skew 500 ppm
Aug 11 05:17:39 dist-test-slave-f7th krb5kdc[5176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425459, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:39.320827 5199 init.cc:377] Logged in from keytab as kudu/127.28.143.190@KRBTEST.COM (short username kudu)
I20260811 05:17:39.321331 5199 webserver.cc:460] Webserver started at http://127.28.143.190:36353/ using document root <none> and password file <none>
I20260811 05:17:39.321549 5199 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:39.321599 5199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:39.321728 5199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:39.322835 5199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-0/data/instance:
uuid: "8fcd48defb83472c87351aeb1f3da757"
format_stamp: "Formatted at 2026-08-11 05:17:39 on dist-test-slave-f7th"
server_key: "7a630e5c78948dafcab880a7c3866f6d"
server_key_iv: "b56b5f4a1f0b837c870d034dd796f161"
server_key_version: "encryptionkey@0"
I20260811 05:17:39.323205 5199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-0/wal/instance:
uuid: "8fcd48defb83472c87351aeb1f3da757"
format_stamp: "Formatted at 2026-08-11 05:17:39 on dist-test-slave-f7th"
server_key: "7a630e5c78948dafcab880a7c3866f6d"
server_key_iv: "b56b5f4a1f0b837c870d034dd796f161"
server_key_version: "encryptionkey@0"
I20260811 05:17:39.324563 5199 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 05:17:39.325600 5224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:39.326108 5199 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 05:17:39.326247 5199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-0/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-0/wal
uuid: "8fcd48defb83472c87351aeb1f3da757"
format_stamp: "Formatted at 2026-08-11 05:17:39 on dist-test-slave-f7th"
server_key: "7a630e5c78948dafcab880a7c3866f6d"
server_key_iv: "b56b5f4a1f0b837c870d034dd796f161"
server_key_version: "encryptionkey@0"
I20260811 05:17:39.326335 5199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:39.347090 5199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:39.349050 5199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:39.349195 5199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:39.353435 5199 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.190:33785
I20260811 05:17:39.353485 5325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.190:33785 every 8 connection(s)
I20260811 05:17:39.353843 5199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-0/data/info.pb
I20260811 05:17:39.354755 5327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:39.356434 5327 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.143.190" port: 33785 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:17:39.357908 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 5199
I20260811 05:17:39.357999 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-0/wal/instance
I20260811 05:17:39.358193 29246 external_mini_cluster.cc:1428] Setting key 5049247652bea785e092aa8de9ac4547
Aug 11 05:17:39 dist-test-slave-f7th krb5kdc[5176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425459, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.28.143.189@KRBTEST.COM; defaulting to no policy
I20260811 05:17:39.366830 5337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:39.357423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45080 (local address 127.28.143.190:33785)
0811 05:17:39.357574 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:39.357578 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:39.357608 (+ 30us) server_negotiation.cc:407] Connection header received
0811 05:17:39.358171 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:39.358173 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:39.358191 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:39.358272 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:39.358839 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.359662 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.360638 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.361100 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.361212 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:39.363621 (+ 2409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:39.363629 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:39.363629 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:39.363637 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:39.365181 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:39.365645 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:39.365648 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:39.365648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:39.365725 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:39.365986 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:39.365986 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:39.365987 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:39.366206 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:39.366268 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:39.366505 (+ 237us) server_negotiation.cc:299] Negotiation successful
0811 05:17:39.366639 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":121,"thread_start_us":56,"threads_started":1}
I20260811 05:17:39.366854 5333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:39.357239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:33785 (local address 127.0.0.1:45080)
0811 05:17:39.357409 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:39.357448 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:39.357953 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:39.358286 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:39.358289 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:39.358578 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:39.358736 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:39.358740 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.359781 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.359783 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:39.360521 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:39.360524 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.361224 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.361226 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:39.361296 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:39.361886 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:17:39.361896 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:39.363496 (+ 1600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:39.365270 (+ 1774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:39.365278 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:39.365282 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:39.365566 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:39.365783 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:39.365784 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:39.365785 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:39.365898 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:39.366278 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:39.366286 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:39.366371 (+ 85us) client_negotiation.cc:769] Sending connection context
0811 05:17:39.366478 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 05:17:39.366719 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":68,"thread_start_us":89,"threads_started":1}
I20260811 05:17:39.367761 5327 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.143.189" port: 35489 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:17:39.368214 5333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:39.368080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.189:35489 (local address 127.0.0.1:57240)
0811 05:17:39.368129 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:39.368186 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.189:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260811 05:17:39.368346 5229 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.143.189:35489: connect: Connection refused (error 111)
I20260811 05:17:39.368813 5333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:39.368713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.189:35489 (local address 127.0.0.1:57256)
0811 05:17:39.368758 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:39.368789 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.189:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260811 05:17:39.368964 5327 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.143.189:35489: Network error: Client connection negotiation failed: client connection to 127.28.143.189:35489: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.189.keytab.
Entry for principal kudu/127.28.143.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.189.keytab.
I20260811 05:17:39.388491 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.189:35489
--webserver_interface=127.28.143.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.189.keytab
--principal=kudu/127.28.143.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:38901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:33785,127.28.143.189:35489,127.28.143.188:40551
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:17:39.418251 5327 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.143.189" port: 35489 } attempt: 1
I20260811 05:17:39.418838 5333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:39.418690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.189:35489 (local address 127.0.0.1:57260)
0811 05:17:39.418756 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:39.418804 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.189:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 05:17:39.419180 5333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:39.419097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.189:35489 (local address 127.0.0.1:57276)
0811 05:17:39.419136 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:39.419162 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.189:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260811 05:17:39.419278 5327 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.143.189:35489: Network error: Client connection negotiation failed: client connection to 127.28.143.189:35489: connect: Connection refused (error 111)
W20260811 05:17:39.479027 5343 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 05:17:39.479274 5343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:39.479290 5343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:39.479312 5343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:39.480567 5343 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:39.480644 5343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:39.480666 5343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:39.480682 5343 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:39.480706 5343 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:39.482370 5343 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:38901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:33785,127.28.143.189:35489,127.28.143.188:40551
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.189.keytab
--principal=kudu/127.28.143.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.189:35489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.28.143.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:17:39.482643 5343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:39.482877 5343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 05:17:39.487428 5327 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.143.189" port: 35489 } attempt: 2
I20260811 05:17:39.488015 5333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:39.487830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.189:35489 (local address 127.0.0.1:57292)
0811 05:17:39.487929 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:39.487974 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.189:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260811 05:17:39.488149 5350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:39.488240 5357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:39.488149 5352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:39.488579 5333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:39.488450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.189:35489 (local address 127.0.0.1:57298)
0811 05:17:39.488516 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:39.488550 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.189:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 05:17:39.488584 5343 server_base.cc:1034] running on GCE node
W20260811 05:17:39.488747 5327 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.143.189:35489: Network error: Client connection negotiation failed: client connection to 127.28.143.189:35489: connect: Connection refused (error 111)
I20260811 05:17:39.488940 5343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:39.489310 5343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:39.490479 5343 hybrid_clock.cc:648] HybridClock initialized: now 1786425459490458 us; error 36 us; skew 500 ppm
Aug 11 05:17:39 dist-test-slave-f7th krb5kdc[5176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425459, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:39.493957 5343 init.cc:377] Logged in from keytab as kudu/127.28.143.189@KRBTEST.COM (short username kudu)
I20260811 05:17:39.494560 5343 webserver.cc:460] Webserver started at http://127.28.143.189:44773/ using document root <none> and password file <none>
I20260811 05:17:39.494865 5343 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:39.494931 5343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:39.495069 5343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:39.496645 5343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-1/data/instance:
uuid: "6c20d365af454fa9906f7653f671acba"
format_stamp: "Formatted at 2026-08-11 05:17:39 on dist-test-slave-f7th"
server_key: "f415edf51dfe7c88af2fa3eefae30ded"
server_key_iv: "a9e7bad3832655fda55f6c0ed86bac9c"
server_key_version: "encryptionkey@0"
I20260811 05:17:39.497100 5343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-1/wal/instance:
uuid: "6c20d365af454fa9906f7653f671acba"
format_stamp: "Formatted at 2026-08-11 05:17:39 on dist-test-slave-f7th"
server_key: "f415edf51dfe7c88af2fa3eefae30ded"
server_key_iv: "a9e7bad3832655fda55f6c0ed86bac9c"
server_key_version: "encryptionkey@0"
I20260811 05:17:39.498937 5343 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:17:39.499861 5374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:39.500062 5343 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:17:39.500165 5343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-1/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-1/wal
uuid: "6c20d365af454fa9906f7653f671acba"
format_stamp: "Formatted at 2026-08-11 05:17:39 on dist-test-slave-f7th"
server_key: "f415edf51dfe7c88af2fa3eefae30ded"
server_key_iv: "a9e7bad3832655fda55f6c0ed86bac9c"
server_key_version: "encryptionkey@0"
I20260811 05:17:39.500245 5343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:39.508332 5343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:39.510710 5343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:39.510841 5343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:39.514878 5343 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.189:35489
I20260811 05:17:39.514937 5459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.189:35489 every 8 connection(s)
I20260811 05:17:39.515303 5343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-1/data/info.pb
I20260811 05:17:39.515669 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 5343
I20260811 05:17:39.515774 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-1/wal/instance
I20260811 05:17:39.516005 29246 external_mini_cluster.cc:1428] Setting key de3fc7df37d456a2850589c4d0c927c7
I20260811 05:17:39.516429 5460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:39.518227 5460 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.143.190" port: 33785 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.28.143.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.143.188@KRBTEST.COM" created.
Aug 11 05:17:39 dist-test-slave-f7th krb5kdc[5176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425459, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.189@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:17:39.537923 5337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:39.519170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45092 (local address 127.28.143.190:33785)
0811 05:17:39.519235 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:39.519237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:39.519589 (+ 352us) server_negotiation.cc:407] Connection header received
0811 05:17:39.520273 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:39.520275 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:39.520294 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:39.520369 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:39.521300 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.522071 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.523635 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.523958 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.524053 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:39.534024 (+ 9971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:39.534027 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:39.534027 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:39.534041 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:39.535790 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:39.536571 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:39.536573 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:39.536573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:39.536697 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:39.537055 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:39.537056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:39.537056 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:39.537222 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:39.537267 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:39.537690 (+ 423us) server_negotiation.cc:299] Negotiation successful
0811 05:17:39.537807 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 05:17:39.538585 5463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:39.519149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:33785 (local address 127.0.0.1:45092)
0811 05:17:39.519438 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:39.519481 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:39.520133 (+ 652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:39.520457 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:39.520462 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:39.520890 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:39.521180 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:39.521186 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.522212 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.522213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:39.523516 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:39.523519 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.524075 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.524076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:39.524257 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:39.524952 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:17:39.524965 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:39.533833 (+ 8868us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:39.535930 (+ 2097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:39.535997 (+ 67us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:39.536003 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:39.536437 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:39.536790 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:39.536791 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:39.536791 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:39.536975 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:39.537314 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:39.537322 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:39.537522 (+ 200us) client_negotiation.cc:769] Sending connection context
0811 05:17:39.538085 (+ 563us) client_negotiation.cc:240] Negotiation successful
0811 05:17:39.538315 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":145,"threads_started":1}
I20260811 05:17:39.539368 5460 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.143.189" port: 35489 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 05:17:39 dist-test-slave-f7th krb5kdc[5176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425459, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.189@KRBTEST.COM for kudu/127.28.143.189@KRBTEST.COM
I20260811 05:17:39.550390 5466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:39.539769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57308 (local address 127.28.143.189:35489)
0811 05:17:39.539987 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:39.539993 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:39.540022 (+ 29us) server_negotiation.cc:407] Connection header received
0811 05:17:39.540085 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:39.540088 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:39.540105 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:39.540195 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:39.540546 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.541733 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.542721 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.543256 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.543337 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:39.546847 (+ 3510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:39.546865 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:39.546865 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:39.546875 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:39.548927 (+ 2052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:39.549408 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:39.549413 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:39.549413 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:39.549482 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:39.549684 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:39.549685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:39.549686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:39.549894 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:39.549980 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:39.550140 (+ 160us) server_negotiation.cc:299] Negotiation successful
0811 05:17:39.550280 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":69,"server-negotiator.queue_time_us":184,"thread_start_us":124,"threads_started":1}
I20260811 05:17:39.550400 5463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:39.539720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.189:35489 (local address 127.0.0.1:57308)
0811 05:17:39.539957 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:39.539969 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:39.540018 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:39.540209 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:39.540210 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:39.540388 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:39.540466 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:39.540468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.541847 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.541849 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:39.542640 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:39.542643 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.543359 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.543360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:39.543434 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:39.543929 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:17:39.543934 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:39.546731 (+ 2797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:39.549048 (+ 2317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:39.549049 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:39.549049 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:39.549299 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:39.549543 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:39.549543 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:39.549544 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:39.549622 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:39.549990 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:39.549991 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:39.550035 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 05:17:39.550097 (+ 62us) client_negotiation.cc:240] Negotiation successful
0811 05:17:39.550214 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.143.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.188.keytab.
Entry for principal kudu/127.28.143.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.188.keytab.
I20260811 05:17:39.551041 5460 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.143.188" port: 40551 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:17:39.551389 5463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:39.551270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:40551 (local address 127.0.0.1:60796)
0811 05:17:39.551314 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:39.551369 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.188:40551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260811 05:17:39.551781 29246 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY_TM7_/build/release/bin/kudu
/tmp/dist-test-taskY_TM7_/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.188:40551
--webserver_interface=127.28.143.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.188.keytab
--principal=kudu/127.28.143.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.143.148:38901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.143.190:33785,127.28.143.189:35489,127.28.143.188:40551
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:39.552760 5376 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.143.188:40551: connect: Connection refused (error 111)
I20260811 05:17:39.553232 5463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:39.553148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:40551 (local address 127.0.0.1:60800)
0811 05:17:39.553185 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:39.553208 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.188:40551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260811 05:17:39.553424 5460 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.143.188:40551: Network error: Client connection negotiation failed: client connection to 127.28.143.188:40551: connect: Connection refused (error 111)
I20260811 05:17:39.579933 5327 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.143.189" port: 35489 } attempt: 3
Aug 11 05:17:39 dist-test-slave-f7th krb5kdc[5176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425459, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for kudu/127.28.143.189@KRBTEST.COM
I20260811 05:17:39.588785 5333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:39.580321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.189:35489 (local address 127.0.0.1:57320)
0811 05:17:39.580399 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:39.580415 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:39.580489 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:39.580691 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:39.580693 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:39.580911 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:39.580990 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:39.580992 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.581971 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.581972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:39.582787 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:39.582790 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.583364 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.583365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:39.583449 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:39.583977 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:17:39.583983 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:39.585669 (+ 1686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:39.587670 (+ 2001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:39.587671 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:39.587672 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:39.587859 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:39.588062 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:39.588062 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:39.588062 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:39.588115 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:39.588413 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:39.588415 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:39.588472 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 05:17:39.588541 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 05:17:39.588707 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 05:17:39.588795 5466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:39.580403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57320 (local address 127.28.143.189:35489)
0811 05:17:39.580451 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:39.580454 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:39.580471 (+ 17us) server_negotiation.cc:407] Connection header received
0811 05:17:39.580578 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:39.580579 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:39.580593 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:39.580681 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:39.581073 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.581870 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.582888 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.583280 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.583360 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:39.585782 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:39.585784 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:39.585785 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:39.585795 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:39.587561 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:39.587929 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:39.587930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:39.587931 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:39.587999 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:39.588167 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:39.588167 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:39.588168 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:39.588317 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:39.588382 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:39.588575 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 05:17:39.588697 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 05:17:39.589540 5327 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.143.188" port: 40551 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:17:39.589903 5333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:39.589795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:40551 (local address 127.0.0.1:60816)
0811 05:17:39.589842 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:39.589874 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.188:40551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260811 05:17:39.590236 5333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:39.590151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:40551 (local address 127.0.0.1:60826)
0811 05:17:39.590193 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:39.590218 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.188:40551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260811 05:17:39.590325 5327 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.143.188:40551: Network error: Client connection negotiation failed: client connection to 127.28.143.188:40551: connect: Connection refused (error 111)
I20260811 05:17:39.602751 5460 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.143.188" port: 40551 } attempt: 1
I20260811 05:17:39.603360 5463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:39.603184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:40551 (local address 127.0.0.1:60830)
0811 05:17:39.603266 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:39.603316 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.188:40551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 05:17:39.603806 5463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:39.603684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:40551 (local address 127.0.0.1:60842)
0811 05:17:39.603748 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:39.603782 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.188:40551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 05:17:39.603927 5460 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.143.188:40551: Network error: Client connection negotiation failed: client connection to 127.28.143.188:40551: connect: Connection refused (error 111)
I20260811 05:17:39.621510 5327 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.143.188" port: 40551 } attempt: 1
I20260811 05:17:39.622193 5333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:39.621966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:40551 (local address 127.0.0.1:60854)
0811 05:17:39.622094 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:39.622153 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.188:40551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260811 05:17:39.622721 5333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:39.622567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:40551 (local address 127.0.0.1:60870)
0811 05:17:39.622648 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:39.622690 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.188:40551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 05:17:39.622860 5327 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.143.188:40551: Network error: Client connection negotiation failed: client connection to 127.28.143.188:40551: connect: Connection refused (error 111)
W20260811 05:17:39.641713 5472 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 05:17:39.641968 5472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:39.641984 5472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:39.642006 5472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:39.643226 5472 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:39.643275 5472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:39.643288 5472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:39.643299 5472 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:39.643311 5472 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:39.644941 5472 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.143.148:38901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.28.143.190:33785,127.28.143.189:35489,127.28.143.188:40551
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/krb5kdc/kudu_127.28.143.188.keytab
--principal=kudu/127.28.143.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.143.188:40551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.28.143.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 03:53:23 UTC on ccycloud.quasar-rvwgcx.root.comops.site
build id 4579567
I20260811 05:17:39.645200 5472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:39.645402 5472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:39.650695 5484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:39.650733 5480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:39.650696 5472 server_base.cc:1034] running on GCE node
W20260811 05:17:39.650695 5487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:39.651304 5472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:39.651690 5472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:39.652839 5472 hybrid_clock.cc:648] HybridClock initialized: now 1786425459652799 us; error 62 us; skew 500 ppm
Aug 11 05:17:39 dist-test-slave-f7th krb5kdc[5176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425459, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:39.655463 5472 init.cc:377] Logged in from keytab as kudu/127.28.143.188@KRBTEST.COM (short username kudu)
I20260811 05:17:39.655902 5472 webserver.cc:460] Webserver started at http://127.28.143.188:34877/ using document root <none> and password file <none>
I20260811 05:17:39.656117 5472 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:39.656159 5472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:39.656251 5472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:39.657356 5472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-2/data/instance:
uuid: "2d01382f893841e28f010fc5c5eb8f29"
format_stamp: "Formatted at 2026-08-11 05:17:39 on dist-test-slave-f7th"
server_key: "a614c2f1892960fa2275553242c2f8e5"
server_key_iv: "89fa1f9edb1e48db7e0fe527ea796772"
server_key_version: "encryptionkey@0"
I20260811 05:17:39.657708 5472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-2/wal/instance:
uuid: "2d01382f893841e28f010fc5c5eb8f29"
format_stamp: "Formatted at 2026-08-11 05:17:39 on dist-test-slave-f7th"
server_key: "a614c2f1892960fa2275553242c2f8e5"
server_key_iv: "89fa1f9edb1e48db7e0fe527ea796772"
server_key_version: "encryptionkey@0"
I20260811 05:17:39.659060 5472 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 05:17:39.659791 5499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:39.659966 5472 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 05:17:39.660059 5472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-2/data,/tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-2/wal
uuid: "2d01382f893841e28f010fc5c5eb8f29"
format_stamp: "Formatted at 2026-08-11 05:17:39 on dist-test-slave-f7th"
server_key: "a614c2f1892960fa2275553242c2f8e5"
server_key_iv: "89fa1f9edb1e48db7e0fe527ea796772"
server_key_version: "encryptionkey@0"
I20260811 05:17:39.660130 5472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:39.672129 5460 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.143.188" port: 40551 } attempt: 2
I20260811 05:17:39.672744 5463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:39.672525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:40551 (local address 127.0.0.1:60884)
0811 05:17:39.672652 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:39.672707 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.188:40551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 05:17:39.673151 5463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:39.673023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:40551 (local address 127.0.0.1:60890)
0811 05:17:39.673089 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:39.673124 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.143.188:40551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 05:17:39.673267 5460 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.143.188:40551: Network error: Client connection negotiation failed: client connection to 127.28.143.188:40551: connect: Connection refused (error 111)
I20260811 05:17:39.675513 5472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:39.677853 5472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:39.678105 5472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:39.682380 5472 rpc_server.cc:307] RPC server started. Bound to: 127.28.143.188:40551
I20260811 05:17:39.682427 5591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.143.188:40551 every 8 connection(s)
I20260811 05:17:39.682787 5472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-2/data/info.pb
I20260811 05:17:39.683759 5592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:39.685423 5592 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.143.190" port: 33785 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:17:39.688364 29246 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY_TM7_/build/release/bin/kudu as pid 5472
I20260811 05:17:39.688465 29246 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY_TM7_/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425159060450-29246-0/minicluster-data/master-2/wal/instance
I20260811 05:17:39.688694 29246 external_mini_cluster.cc:1428] Setting key 8c3ee8dba3034ad0085f7f1868e8d2cf
Aug 11 05:17:39 dist-test-slave-f7th krb5kdc[5176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425459, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.188@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
Aug 11 05:17:39 dist-test-slave-f7th krb5kdc[5176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.143.188@KRBTEST.COM
I20260811 05:17:39.695045 5337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:39.686238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45104 (local address 127.28.143.190:33785)
0811 05:17:39.686305 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:39.686307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:39.686545 (+ 238us) server_negotiation.cc:407] Connection header received
0811 05:17:39.687078 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:39.687079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:39.687091 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:39.687151 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:39.688004 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.688567 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.689875 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.690070 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.690128 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:39.692331 (+ 2203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:39.692332 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:39.692333 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:39.692342 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:39.693592 (+ 1250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:39.693988 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:39.693989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:39.693990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:39.694044 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:39.694303 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:39.694304 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:39.694304 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:39.694466 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:39.694531 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:39.694884 (+ 353us) server_negotiation.cc:299] Negotiation successful
0811 05:17:39.694972 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 05:17:39.695207 5595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:39.686198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.190:33785 (local address 127.0.0.1:45104)
0811 05:17:39.686404 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:39.686443 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:39.686964 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:39.687281 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:39.687286 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:39.687647 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:39.687855 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:39.687860 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.688682 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.688684 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:39.689792 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:39.689794 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.690144 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.690144 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:39.690268 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:39.690757 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:17:39.690766 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:39.692244 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:39.693672 (+ 1428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:39.693680 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:39.693682 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:39.693913 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:39.694114 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:39.694115 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:39.694116 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:39.694237 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:39.694522 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:39.694530 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:39.694687 (+ 157us) client_negotiation.cc:769] Sending connection context
0811 05:17:39.694827 (+ 140us) client_negotiation.cc:240] Negotiation successful
0811 05:17:39.695004 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":107,"threads_started":1}
I20260811 05:17:39.695762 5592 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.143.189" port: 35489 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:17:39.697788 5598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:39.689116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60894 (local address 127.28.143.188:40551)
0811 05:17:39.689285 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:39.689290 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:39.689313 (+ 23us) server_negotiation.cc:407] Connection header received
0811 05:17:39.689355 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:39.689357 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:39.689379 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:39.689464 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:39.690108 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.690810 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.691509 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.691857 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.691925 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:39.694476 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:39.694481 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:39.694481 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:39.694487 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:39.696512 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:39.696982 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:39.696985 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:39.696985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:39.697090 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:39.697269 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:39.697270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:39.697270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:39.697430 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:39.697562 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:39.697648 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 05:17:39.697716 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":133,"thread_start_us":97,"threads_started":1}
I20260811 05:17:39.698015 5327 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.143.188" port: 40551 } attempt: 2
Aug 11 05:17:39 dist-test-slave-f7th krb5kdc[5176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425459, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.188@KRBTEST.COM for kudu/127.28.143.189@KRBTEST.COM
Aug 11 05:17:39 dist-test-slave-f7th krb5kdc[5176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425459, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.190@KRBTEST.COM for kudu/127.28.143.188@KRBTEST.COM
Aug 11 05:17:39 dist-test-slave-f7th krb5kdc[5176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.143.190@KRBTEST.COM
I20260811 05:17:39.703281 5466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:39.696094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57330 (local address 127.28.143.189:35489)
0811 05:17:39.696162 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:39.696164 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:39.696172 (+ 8us) server_negotiation.cc:407] Connection header received
0811 05:17:39.696195 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:39.696196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:39.696206 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:39.696250 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:39.696584 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.697271 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.698137 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.698439 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.698503 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:39.700517 (+ 2014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:39.700518 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:39.700519 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:39.700527 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:39.702037 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:39.702410 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:39.702412 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:39.702412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:39.702488 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:39.702728 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:39.702729 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:39.702730 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:39.702859 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:39.702920 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:39.703106 (+ 186us) server_negotiation.cc:299] Negotiation successful
0811 05:17:39.703188 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 05:17:39.703565 5595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:39.696019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.189:35489 (local address 127.0.0.1:57330)
0811 05:17:39.696061 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:39.696071 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:39.696106 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:39.696258 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:39.696258 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:39.696481 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:39.696516 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:39.696518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.697327 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.697328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:39.698032 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:39.698035 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.698561 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.698562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:39.698625 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:39.699010 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:17:39.699016 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:39.700439 (+ 1423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:39.702125 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:39.702126 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:39.702126 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:39.702328 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:39.702570 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:39.702571 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:39.702572 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:39.702636 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:39.702934 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:39.702935 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:39.702983 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 05:17:39.703378 (+ 395us) client_negotiation.cc:240] Negotiation successful
0811 05:17:39.703479 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260811 05:17:39.704082 5592 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.143.188" port: 40551 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:17:39.706475 5333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:39.698265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:40551 (local address 127.0.0.1:60902)
0811 05:17:39.698312 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:39.698321 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:39.698355 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:39.698537 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:39.698538 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:39.698699 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:39.698742 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:39.698744 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.699541 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.699543 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:39.700311 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:39.700314 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.700833 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.700834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:39.700896 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:39.701453 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:17:39.701457 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:39.702910 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:39.705480 (+ 2570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:39.705481 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:39.705481 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:39.705716 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:39.705884 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:39.705884 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:39.705885 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:39.705940 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:39.706191 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:39.706191 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:39.706230 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 05:17:39.706297 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 05:17:39.706380 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260811 05:17:39.706482 5598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:39.698369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60902 (local address 127.28.143.188:40551)
0811 05:17:39.698402 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:39.698403 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:39.698411 (+ 8us) server_negotiation.cc:407] Connection header received
0811 05:17:39.698427 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:39.698428 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:39.698437 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:39.698524 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:39.698809 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.699471 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.700400 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.700754 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.700814 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:39.702972 (+ 2158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:39.702974 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:39.702974 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:39.702980 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:39.705385 (+ 2405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:39.705774 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:39.705775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:39.705775 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:39.705830 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:39.705997 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:39.705998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:39.705998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:39.706121 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:39.706190 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:39.706325 (+ 135us) server_negotiation.cc:299] Negotiation successful
0811 05:17:39.706398 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260811 05:17:39.706555 5337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:39.699019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45108 (local address 127.28.143.190:33785)
0811 05:17:39.699059 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:39.699061 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:39.699069 (+ 8us) server_negotiation.cc:407] Connection header received
0811 05:17:39.699106 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:39.699107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:39.699116 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:39.699159 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:39.699690 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.700131 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.701054 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.701315 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.701382 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:39.703694 (+ 2312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:39.703696 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:39.703696 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:39.703703 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:39.705062 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:39.705468 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:39.705469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:39.705469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:39.705538 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:39.705716 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:39.705717 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:39.705717 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:39.705846 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:39.706043 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:39.706433 (+ 390us) server_negotiation.cc:299] Negotiation successful
0811 05:17:39.706492 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 05:17:39.708638 5327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fcd48defb83472c87351aeb1f3da757: Bootstrap starting.
I20260811 05:17:39.709425 5327 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8fcd48defb83472c87351aeb1f3da757: Neither blocks nor log segments found. Creating new log.
Aug 11 05:17:39 dist-test-slave-f7th krb5kdc[5176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425459, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.188@KRBTEST.COM for kudu/127.28.143.188@KRBTEST.COM
I20260811 05:17:39.709721 5327 log.cc:824] T 00000000000000000000000000000000 P 8fcd48defb83472c87351aeb1f3da757: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:39.710701 5327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fcd48defb83472c87351aeb1f3da757: No bootstrap required, opened a new log
I20260811 05:17:39.712420 5595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:39.705207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:40551 (local address 127.0.0.1:60904)
0811 05:17:39.705230 (+ 23us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:39.705239 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:39.705281 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:39.705408 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:39.705409 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:39.705552 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:39.705590 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:39.705592 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.707038 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.707039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:39.707571 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:39.707574 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.708025 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.708025 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:39.708063 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:39.708406 (+ 343us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:17:39.708410 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:39.709784 (+ 1374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:39.711453 (+ 1669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:39.711454 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:39.711454 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:39.711608 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:39.711767 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:39.711768 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:39.711768 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:39.711820 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:39.712117 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:39.712117 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:39.712153 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 05:17:39.712216 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 05:17:39.712316 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14}
I20260811 05:17:39.712517 5602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:39.704414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60904 (local address 127.28.143.188:40551)
0811 05:17:39.705196 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:39.705198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:39.705268 (+ 70us) server_negotiation.cc:407] Connection header received
0811 05:17:39.705320 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:39.705321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:39.705331 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:39.705386 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:39.705653 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.706955 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.707649 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.707940 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.708030 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:39.709860 (+ 1830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:39.709861 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:39.709861 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:39.709867 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:39.711391 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:39.711665 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:39.711666 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:39.711666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:39.711719 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:39.711867 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:39.711868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:39.711868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:39.712032 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:39.712095 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:39.712241 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 05:17:39.712367 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":750,"thread_start_us":90,"threads_started":1}
I20260811 05:17:39.712594 5327 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8fcd48defb83472c87351aeb1f3da757 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fcd48defb83472c87351aeb1f3da757" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 33785 } } peers { permanent_uuid: "6c20d365af454fa9906f7653f671acba" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 35489 } } peers { permanent_uuid: "2d01382f893841e28f010fc5c5eb8f29" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 40551 } }
I20260811 05:17:39.712783 5327 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8fcd48defb83472c87351aeb1f3da757 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fcd48defb83472c87351aeb1f3da757, State: Initialized, Role: FOLLOWER
I20260811 05:17:39.712865 5327 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8fcd48defb83472c87351aeb1f3da757 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fcd48defb83472c87351aeb1f3da757" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 33785 } } peers { permanent_uuid: "6c20d365af454fa9906f7653f671acba" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 35489 } } peers { permanent_uuid: "2d01382f893841e28f010fc5c5eb8f29" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 40551 } }
I20260811 05:17:39.713205 5604 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fcd48defb83472c87351aeb1f3da757 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fcd48defb83472c87351aeb1f3da757" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 33785 } } peers { permanent_uuid: "6c20d365af454fa9906f7653f671acba" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 35489 } } peers { permanent_uuid: "2d01382f893841e28f010fc5c5eb8f29" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 40551 } } }
I20260811 05:17:39.713274 5327 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8fcd48defb83472c87351aeb1f3da757 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:39.713383 5604 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fcd48defb83472c87351aeb1f3da757 [sys.catalog]: This master's current role is: FOLLOWER
Aug 11 05:17:39 dist-test-slave-f7th krb5kdc[5176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.143.189@KRBTEST.COM
I20260811 05:17:39.714166 5592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d01382f893841e28f010fc5c5eb8f29: Bootstrap starting.
I20260811 05:17:39.714848 5592 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d01382f893841e28f010fc5c5eb8f29: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:39.715143 5592 log.cc:824] T 00000000000000000000000000000000 P 2d01382f893841e28f010fc5c5eb8f29: Log is configured to *not* fsync() on all Append() calls
W20260811 05:17:39.715790 5626 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8fcd48defb83472c87351aeb1f3da757: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:17:39.715898 5626 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:17:39.715939 5592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d01382f893841e28f010fc5c5eb8f29: No bootstrap required, opened a new log
I20260811 05:17:39.717684 5466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:39.709051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57334 (local address 127.28.143.189:35489)
0811 05:17:39.709091 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:39.709093 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:39.709103 (+ 10us) server_negotiation.cc:407] Connection header received
0811 05:17:39.709132 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:39.709133 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:39.709142 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:39.709201 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:39.709750 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.710407 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.711322 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.711609 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.711680 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:39.714436 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:39.714438 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:39.714438 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:39.714446 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:39.716084 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:39.716469 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:39.716470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:39.716471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:39.716530 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:39.716746 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:39.716747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:39.716748 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:39.716872 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:39.717092 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:39.717495 (+ 403us) server_negotiation.cc:299] Negotiation successful
0811 05:17:39.717582 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 05:17:39.718034 5592 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d01382f893841e28f010fc5c5eb8f29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fcd48defb83472c87351aeb1f3da757" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 33785 } } peers { permanent_uuid: "6c20d365af454fa9906f7653f671acba" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 35489 } } peers { permanent_uuid: "2d01382f893841e28f010fc5c5eb8f29" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 40551 } }
I20260811 05:17:39.718197 5592 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d01382f893841e28f010fc5c5eb8f29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d01382f893841e28f010fc5c5eb8f29, State: Initialized, Role: FOLLOWER
I20260811 05:17:39.718289 5592 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d01382f893841e28f010fc5c5eb8f29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fcd48defb83472c87351aeb1f3da757" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 33785 } } peers { permanent_uuid: "6c20d365af454fa9906f7653f671acba" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 35489 } } peers { permanent_uuid: "2d01382f893841e28f010fc5c5eb8f29" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 40551 } }
I20260811 05:17:39.718622 5592 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d01382f893841e28f010fc5c5eb8f29 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:39.718582 5631 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d01382f893841e28f010fc5c5eb8f29 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fcd48defb83472c87351aeb1f3da757" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 33785 } } peers { permanent_uuid: "6c20d365af454fa9906f7653f671acba" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 35489 } } peers { permanent_uuid: "2d01382f893841e28f010fc5c5eb8f29" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 40551 } } }
I20260811 05:17:39.718679 5631 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d01382f893841e28f010fc5c5eb8f29 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:17:39.716449 5199 master_runner.cc:186] Error getting master registration for 127.28.143.189:35489: OK, instance_id { permanent_uuid: "6c20d365af454fa9906f7653f671acba" instance_seqno: 1786425459513739 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 05:17:39.719113 5199 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 05:17:39.720402 29246 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260811 05:17:39.720561 5650 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2d01382f893841e28f010fc5c5eb8f29: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:17:39.721222 5650 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:17:39.721489 5275 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "8fcd48defb83472c87351aeb1f3da757"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45108
I20260811 05:17:39.721606 5275 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8fcd48defb83472c87351aeb1f3da757 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 05:17:39.721655 5275 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8fcd48defb83472c87351aeb1f3da757 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:39.722548 5275 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8fcd48defb83472c87351aeb1f3da757 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fcd48defb83472c87351aeb1f3da757" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 33785 } } peers { permanent_uuid: "6c20d365af454fa9906f7653f671acba" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 35489 } } peers { permanent_uuid: "2d01382f893841e28f010fc5c5eb8f29" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 40551 } }
I20260811 05:17:39.722788 5275 leader_election.cc:288] T 00000000000000000000000000000000 P 8fcd48defb83472c87351aeb1f3da757 [CANDIDATE]: Term 1 election: Requested vote from peers 6c20d365af454fa9906f7653f671acba (127.28.143.189:35489), 2d01382f893841e28f010fc5c5eb8f29 (127.28.143.188:40551)
I20260811 05:17:39.723182 5553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8fcd48defb83472c87351aeb1f3da757" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "2d01382f893841e28f010fc5c5eb8f29"
I20260811 05:17:39.723327 5553 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d01382f893841e28f010fc5c5eb8f29 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:39.724277 5553 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2d01382f893841e28f010fc5c5eb8f29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fcd48defb83472c87351aeb1f3da757 in term 1.
I20260811 05:17:39.724632 5226 leader_election.cc:302] T 00000000000000000000000000000000 P 8fcd48defb83472c87351aeb1f3da757 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d01382f893841e28f010fc5c5eb8f29, 8fcd48defb83472c87351aeb1f3da757; no voters:
I20260811 05:17:39.724815 5604 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8fcd48defb83472c87351aeb1f3da757 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:39.724992 5604 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8fcd48defb83472c87351aeb1f3da757 [term 1 LEADER]: Becoming Leader. State: Replica: 8fcd48defb83472c87351aeb1f3da757, State: Running, Role: LEADER
I20260811 05:17:39.725085 5604 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8fcd48defb83472c87351aeb1f3da757 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fcd48defb83472c87351aeb1f3da757" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 33785 } } peers { permanent_uuid: "6c20d365af454fa9906f7653f671acba" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 35489 } } peers { permanent_uuid: "2d01382f893841e28f010fc5c5eb8f29" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 40551 } }
I20260811 05:17:39.725526 5604 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fcd48defb83472c87351aeb1f3da757 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8fcd48defb83472c87351aeb1f3da757. Latest consensus state: current_term: 1 leader_uuid: "8fcd48defb83472c87351aeb1f3da757" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fcd48defb83472c87351aeb1f3da757" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 33785 } } peers { permanent_uuid: "6c20d365af454fa9906f7653f671acba" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 35489 } } peers { permanent_uuid: "2d01382f893841e28f010fc5c5eb8f29" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 40551 } } }
I20260811 05:17:39.725612 5604 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fcd48defb83472c87351aeb1f3da757 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:39.723408 5472 master_runner.cc:186] Error getting master registration for 127.28.143.189:35489: OK, instance_id { permanent_uuid: "6c20d365af454fa9906f7653f671acba" instance_seqno: 1786425459513739 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 05:17:39.725826 5472 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 05:17:39.730005 5669 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:17:39.730338 5669 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:17:39.738507 5466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:39.725982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57350 (local address 127.28.143.189:35489)
0811 05:17:39.726028 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:39.726030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:39.729317 (+ 3287us) server_negotiation.cc:407] Connection header received
0811 05:17:39.729673 (+ 356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:39.729675 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:39.729703 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:39.729738 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:39.731097 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.731878 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.733261 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.733971 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.734042 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:39.735599 (+ 1557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:39.735602 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:39.735602 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:39.735610 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:39.737301 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:39.737722 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:39.737724 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:39.737724 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:39.737792 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:39.737985 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:39.737986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:39.737987 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:39.738106 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:39.738212 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:39.738320 (+ 108us) server_negotiation.cc:299] Negotiation successful
0811 05:17:39.738410 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 05:17:39.746030 5337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:39.731274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45116 (local address 127.28.143.190:33785)
0811 05:17:39.731325 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:39.731327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:39.738553 (+ 7226us) server_negotiation.cc:407] Connection header received
0811 05:17:39.738744 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:39.738745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:39.738758 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:39.738786 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:39.739422 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.739926 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.740923 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.741128 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.741188 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:39.742671 (+ 1483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:39.742674 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:39.742674 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:39.742683 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:39.744266 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:39.744785 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:39.744786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:39.744787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:39.744998 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:39.745304 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:39.745305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:39.745306 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:39.745474 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:39.745596 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:39.745822 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 05:17:39.745920 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 05:17:39.746311 5553 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2d01382f893841e28f010fc5c5eb8f29 [term 1 FOLLOWER]: Refusing update from remote peer 8fcd48defb83472c87351aeb1f3da757: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:17:39.746802 5663 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8fcd48defb83472c87351aeb1f3da757 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d01382f893841e28f010fc5c5eb8f29" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 40551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:17:39.748792 5602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:39.728896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60916 (local address 127.28.143.188:40551)
0811 05:17:39.729263 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:39.729266 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:39.731027 (+ 1761us) server_negotiation.cc:407] Connection header received
0811 05:17:39.731098 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:39.731100 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:39.731115 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:39.731180 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:39.734691 (+ 3511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.735495 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.736556 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.736932 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.736977 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:39.745286 (+ 8309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:39.745288 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:39.745289 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:39.745298 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:39.747515 (+ 2217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:39.747934 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:39.747936 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:39.747937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:39.748009 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:39.748196 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:39.748197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:39.748197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:39.748332 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:39.748427 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:39.748596 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 05:17:39.748704 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313}
I20260811 05:17:39.749230 5631 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d01382f893841e28f010fc5c5eb8f29 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8fcd48defb83472c87351aeb1f3da757. Latest consensus state: current_term: 1 leader_uuid: "8fcd48defb83472c87351aeb1f3da757" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fcd48defb83472c87351aeb1f3da757" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 33785 } } peers { permanent_uuid: "6c20d365af454fa9906f7653f671acba" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 35489 } } peers { permanent_uuid: "2d01382f893841e28f010fc5c5eb8f29" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 40551 } } }
I20260811 05:17:39.749326 5631 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d01382f893841e28f010fc5c5eb8f29 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:17:39.754411 5663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fcd48defb83472c87351aeb1f3da757 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8fcd48defb83472c87351aeb1f3da757" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fcd48defb83472c87351aeb1f3da757" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 33785 } } peers { permanent_uuid: "6c20d365af454fa9906f7653f671acba" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 35489 } } peers { permanent_uuid: "2d01382f893841e28f010fc5c5eb8f29" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 40551 } } }
I20260811 05:17:39.754540 5663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fcd48defb83472c87351aeb1f3da757 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:39.755200 5631 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d01382f893841e28f010fc5c5eb8f29 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8fcd48defb83472c87351aeb1f3da757" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fcd48defb83472c87351aeb1f3da757" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 33785 } } peers { permanent_uuid: "6c20d365af454fa9906f7653f671acba" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 35489 } } peers { permanent_uuid: "2d01382f893841e28f010fc5c5eb8f29" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 40551 } } }
I20260811 05:17:39.755307 5631 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d01382f893841e28f010fc5c5eb8f29 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:17:39.755507 5669 catalog_manager.cc:1347] Generated new cluster ID: 7d31255a764f499d83dede14eb025c18
I20260811 05:17:39.755633 5675 mvcc.cc:205] Tried to move back new op lower bound from 7317198683116105728 to 7317198683034697728. Current Snapshot: MvccSnapshot[applied={T|T < 7317198683116105728}]
I20260811 05:17:39.755659 5669 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:17:39.764398 5460 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.143.188" port: 40551 } attempt: 3
Aug 11 05:17:39 dist-test-slave-f7th krb5kdc[5176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425459, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.143.189@KRBTEST.COM for kudu/127.28.143.188@KRBTEST.COM
I20260811 05:17:39.772944 5463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:39.764752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.143.188:40551 (local address 127.0.0.1:60932)
0811 05:17:39.764857 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:39.764869 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:39.764911 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:39.765098 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:39.765099 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:39.765298 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:39.765349 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:39.765351 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.766321 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.766322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:39.767164 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:39.767167 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.767692 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.767693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:39.767751 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:39.768260 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:17:39.768266 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:39.769876 (+ 1610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:39.771770 (+ 1894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:39.771771 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:39.771772 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:39.772017 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:39.772226 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:39.772227 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:39.772227 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:39.772287 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:39.772584 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:39.772585 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:39.772653 (+ 68us) client_negotiation.cc:769] Sending connection context
0811 05:17:39.772718 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 05:17:39.772834 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260811 05:17:39.773316 5602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:39.764856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60932 (local address 127.28.143.188:40551)
0811 05:17:39.764897 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:39.764899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:39.764910 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:17:39.764992 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:39.764993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:39.765004 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:39.765046 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:39.765435 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.766219 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.767264 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:39.767575 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:39.767631 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:39.769982 (+ 2351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:39.769984 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:39.769984 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:39.769992 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:39.771670 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:39.772093 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:39.772095 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:39.772095 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:39.772159 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:39.772348 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:39.772349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:39.772349 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:39.772489 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:39.772557 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:39.773140 (+ 583us) server_negotiation.cc:299] Negotiation successful
0811 05:17:39.773227 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 05:17:39.773398 5669 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:17:39.774214 5669 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:17:39.774912 5460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c20d365af454fa9906f7653f671acba: Bootstrap starting.
I20260811 05:17:39.775766 5460 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c20d365af454fa9906f7653f671acba: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:39.776101 5460 log.cc:824] T 00000000000000000000000000000000 P 6c20d365af454fa9906f7653f671acba: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:39.776993 5460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c20d365af454fa9906f7653f671acba: No bootstrap required, opened a new log
I20260811 05:17:39.779079 5460 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c20d365af454fa9906f7653f671acba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fcd48defb83472c87351aeb1f3da757" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 33785 } } peers { permanent_uuid: "6c20d365af454fa9906f7653f671acba" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 35489 } } peers { permanent_uuid: "2d01382f893841e28f010fc5c5eb8f29" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 40551 } }
I20260811 05:17:39.779250 5460 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c20d365af454fa9906f7653f671acba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c20d365af454fa9906f7653f671acba, State: Initialized, Role: FOLLOWER
I20260811 05:17:39.779340 5460 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c20d365af454fa9906f7653f671acba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fcd48defb83472c87351aeb1f3da757" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 33785 } } peers { permanent_uuid: "6c20d365af454fa9906f7653f671acba" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 35489 } } peers { permanent_uuid: "2d01382f893841e28f010fc5c5eb8f29" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 40551 } }
I20260811 05:17:39.779706 5460 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c20d365af454fa9906f7653f671acba [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:39.779772 5421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8fcd48defb83472c87351aeb1f3da757" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6c20d365af454fa9906f7653f671acba"
I20260811 05:17:39.779913 5421 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c20d365af454fa9906f7653f671acba [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:39.780819 5421 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6c20d365af454fa9906f7653f671acba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fcd48defb83472c87351aeb1f3da757 in term 1.
I20260811 05:17:39.781064 5420 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6c20d365af454fa9906f7653f671acba [term 1 FOLLOWER]: Refusing update from remote peer 8fcd48defb83472c87351aeb1f3da757: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:17:39.781311 5682 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8fcd48defb83472c87351aeb1f3da757 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c20d365af454fa9906f7653f671acba" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 35489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:17:39.783231 5706 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c20d365af454fa9906f7653f671acba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8fcd48defb83472c87351aeb1f3da757" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fcd48defb83472c87351aeb1f3da757" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 33785 } } peers { permanent_uuid: "6c20d365af454fa9906f7653f671acba" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 35489 } } peers { permanent_uuid: "2d01382f893841e28f010fc5c5eb8f29" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 40551 } } }
I20260811 05:17:39.783939 5706 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c20d365af454fa9906f7653f671acba [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:17:39.784016 5706 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c20d365af454fa9906f7653f671acba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8fcd48defb83472c87351aeb1f3da757. Latest consensus state: current_term: 1 leader_uuid: "8fcd48defb83472c87351aeb1f3da757" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fcd48defb83472c87351aeb1f3da757" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 33785 } } peers { permanent_uuid: "6c20d365af454fa9906f7653f671acba" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 35489 } } peers { permanent_uuid: "2d01382f893841e28f010fc5c5eb8f29" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 40551 } } }
I20260811 05:17:39.784063 5706 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c20d365af454fa9906f7653f671acba [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:17:39.784090 5706 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c20d365af454fa9906f7653f671acba [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8fcd48defb83472c87351aeb1f3da757" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fcd48defb83472c87351aeb1f3da757" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 33785 } } peers { permanent_uuid: "6c20d365af454fa9906f7653f671acba" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 35489 } } peers { permanent_uuid: "2d01382f893841e28f010fc5c5eb8f29" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 40551 } } }
I20260811 05:17:39.784135 5706 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c20d365af454fa9906f7653f671acba [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:17:39.785611 5709 mvcc.cc:205] Tried to move back new op lower bound from 7317198683216916480 to 7317198683034697728. Current Snapshot: MvccSnapshot[applied={T|T < 7317198683116105728}]
I20260811 05:17:39.788038 5663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fcd48defb83472c87351aeb1f3da757 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8fcd48defb83472c87351aeb1f3da757" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fcd48defb83472c87351aeb1f3da757" member_type: VOTER last_known_addr { host: "127.28.143.190" port: 33785 } } peers { permanent_uuid: "6c20d365af454fa9906f7653f671acba" member_type: VOTER last_known_addr { host: "127.28.143.189" port: 35489 } } peers { permanent_uuid: "2d01382f893841e28f010fc5c5eb8f29" member_type: VOTER last_known_addr { host: "127.28.143.188" port: 40551 } } }
I20260811 05:17:39.788151 5663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fcd48defb83472c87351aeb1f3da757 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:39.788208 5669 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8fcd48defb83472c87351aeb1f3da757: Generated new TSK 0
I20260811 05:17:39.788401 5669 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 05:17:39.788398 5714 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6c20d365af454fa9906f7653f671acba: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:17:39.788460 5714 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260811 05:17:39.811110 5466 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:17:39.809540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57362 (local address 127.28.143.189:35489)
0811 05:17:39.809593 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:39.809595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:39.810315 (+ 720us) server_negotiation.cc:407] Connection header received
0811 05:17:39.810424 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:39.810425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:39.810437 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:39.810468 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:39.811031 (+ 563us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57362: BlockingRecv error: recv got EOF from 127.0.0.1:57362 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260811 05:17:39.811956 5602 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:17:39.810024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60936 (local address 127.28.143.188:40551)
0811 05:17:39.810069 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:39.810071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:39.811392 (+ 1321us) server_negotiation.cc:407] Connection header received
0811 05:17:39.811491 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:39.811492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:39.811503 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:39.811531 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:39.811869 (+ 338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60936: BlockingRecv error: recv got EOF from 127.0.0.1:60936 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260811 05:17:39.813118 5337 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:17:39.812333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45122 (local address 127.28.143.190:33785)
0811 05:17:39.812388 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:39.812390 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:39.812518 (+ 128us) server_negotiation.cc:407] Connection header received
0811 05:17:39.812610 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:39.812646 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:39.812658 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:39.812689 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:39.813047 (+ 358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45122: BlockingRecv error: recv got EOF from 127.0.0.1:45122 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260811 05:17:39.821568 5466 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:17:39.818902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57372 (local address 127.28.143.189:35489)
0811 05:17:39.818966 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:39.818969 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:39.820821 (+ 1852us) server_negotiation.cc:407] Connection header received
0811 05:17:39.820962 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:39.820964 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:39.820989 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:39.821022 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:39.821484 (+ 462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57372: BlockingRecv error: recv got EOF from 127.0.0.1:57372 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260811 05:17:39.824977 5466 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:17:39.823976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57378 (local address 127.28.143.189:35489)
0811 05:17:39.824019 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:39.824021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:39.824323 (+ 302us) server_negotiation.cc:407] Connection header received
0811 05:17:39.824427 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:39.824428 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:39.824440 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:39.824482 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:39.824914 (+ 432us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57378: BlockingRecv error: recv got EOF from 127.0.0.1:57378 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260811 05:17:39.829430 5466 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:17:39.828325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57382 (local address 127.28.143.189:35489)
0811 05:17:39.828379 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:39.828381 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:39.828764 (+ 383us) server_negotiation.cc:407] Connection header received
0811 05:17:39.828870 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:39.828871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:39.828884 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:39.828913 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:39.829366 (+ 453us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57382: BlockingRecv error: recv got EOF from 127.0.0.1:57382 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260811 05:17:39.835932 5466 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:17:39.834909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57386 (local address 127.28.143.189:35489)
0811 05:17:39.834959 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:39.834961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:39.835290 (+ 329us) server_negotiation.cc:407] Connection header received
0811 05:17:39.835398 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:39.835400 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:39.835413 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:39.835444 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:39.835865 (+ 421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57386: BlockingRecv error: recv got EOF from 127.0.0.1:57386 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260811 05:17:39.846585 5466 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:17:39.845457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57398 (local address 127.28.143.189:35489)
0811 05:17:39.845507 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:39.845510 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:39.845834 (+ 324us) server_negotiation.cc:407] Connection header received
0811 05:17:39.845947 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:39.845948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:39.845961 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:39.845993 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:39.846505 (+ 512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57398: BlockingRecv error: recv got EOF from 127.0.0.1:57398 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260811 05:17:39.877890 5466 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:17:39.877009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57406 (local address 127.28.143.189:35489)
0811 05:17:39.877060 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:39.877062 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:39.877237 (+ 175us) server_negotiation.cc:407] Connection header received
0811 05:17:39.877344 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:39.877346 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:39.877359 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:39.877390 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:39.877824 (+ 434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57406: BlockingRecv error: recv got EOF from 127.0.0.1:57406 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260811 05:17:39.912523 5466 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:17:39.911473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57422 (local address 127.28.143.189:35489)
0811 05:17:39.911527 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:39.911529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:39.911850 (+ 321us) server_negotiation.cc:407] Connection header received
0811 05:17:39.911973 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:39.911975 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:39.911988 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:39.912019 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:39.912455 (+ 436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57422: BlockingRecv error: recv got EOF from 127.0.0.1:57422 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260811 05:17:39.978602 5466 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:17:39.977947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57426 (local address 127.28.143.189:35489)
0811 05:17:39.978008 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:39.978010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:39.978055 (+ 45us) server_negotiation.cc:407] Connection header received
0811 05:17:39.978131 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:39.978132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:39.978144 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:39.978193 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:39.978539 (+ 346us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57426: BlockingRecv error: recv got EOF from 127.0.0.1:57426 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260811 05:17:40.108914 5466 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:17:40.108151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57434 (local address 127.28.143.189:35489)
0811 05:17:40.108238 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:40.108240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:40.108287 (+ 47us) server_negotiation.cc:407] Connection header received
0811 05:17:40.108370 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:40.108371 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:40.108383 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:40.108432 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:40.108852 (+ 420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57434: BlockingRecv error: recv got EOF from 127.0.0.1:57434 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260811 05:17:40.366948 5466 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:17:40.366182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57438 (local address 127.28.143.189:35489)
0811 05:17:40.366261 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:40.366263 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:40.366275 (+ 12us) server_negotiation.cc:407] Connection header received
0811 05:17:40.366331 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:40.366332 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:40.366343 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:40.366417 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:40.366827 (+ 410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57438: BlockingRecv error: recv got EOF from 127.0.0.1:57438 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 05:17:40.721724 5650 catalog_manager.cc:1259] Loaded cluster ID: 7d31255a764f499d83dede14eb025c18
I20260811 05:17:40.721810 5650 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2d01382f893841e28f010fc5c5eb8f29: loading cluster ID for follower catalog manager: success
I20260811 05:17:40.722708 5650 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2d01382f893841e28f010fc5c5eb8f29: acquiring CA information for follower catalog manager: success
I20260811 05:17:40.723042 5650 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2d01382f893841e28f010fc5c5eb8f29: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 05:17:40.788981 5714 catalog_manager.cc:1259] Loaded cluster ID: 7d31255a764f499d83dede14eb025c18
I20260811 05:17:40.789069 5714 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6c20d365af454fa9906f7653f671acba: loading cluster ID for follower catalog manager: success
I20260811 05:17:40.789937 5714 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6c20d365af454fa9906f7653f671acba: acquiring CA information for follower catalog manager: success
I20260811 05:17:40.790269 5714 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6c20d365af454fa9906f7653f671acba: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 05:17:40.883074 5844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:40.880237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48798 (local address 127.28.143.189:35489)
0811 05:17:40.880358 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:40.880360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:40.880373 (+ 13us) server_negotiation.cc:407] Connection header received
0811 05:17:40.880405 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:40.880406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:40.880462 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:40.880561 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:17:40.880743 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:40.881322 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:40.882206 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:40.882430 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:40.882511 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:40.882706 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:17:40.882838 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:17:40.882959 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:40.882975 (+ 16us) server_negotiation.cc:299] Negotiation successful
0811 05:17:40.882998 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":61,"threads_started":1}
I20260811 05:17:40.888166 5854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:40.885283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55186 (local address 127.28.143.188:40551)
0811 05:17:40.885461 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:40.885463 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:40.885473 (+ 10us) server_negotiation.cc:407] Connection header received
0811 05:17:40.885514 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:40.885515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:40.885558 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:40.885640 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:17:40.885820 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:40.886345 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:40.887237 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:40.887458 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:40.887526 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:40.887721 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:17:40.887849 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:17:40.887919 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:40.888003 (+ 84us) server_negotiation.cc:299] Negotiation successful
0811 05:17:40.888066 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":62,"threads_started":1}
I20260811 05:17:40.893610 5844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:40.890288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48808 (local address 127.28.143.189:35489)
0811 05:17:40.890338 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:40.890340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:40.890665 (+ 325us) server_negotiation.cc:407] Connection header received
0811 05:17:40.890793 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:40.890795 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:40.890811 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:40.890865 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:17:40.890980 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:40.891491 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:40.892752 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:40.893054 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:40.893128 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:40.893324 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:17:40.893363 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:17:40.893385 (+ 22us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:40.893488 (+ 103us) server_negotiation.cc:299] Negotiation successful
0811 05:17:40.893526 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 05:17:40.893602 5854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:40.890341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55200 (local address 127.28.143.188:40551)
0811 05:17:40.890375 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:40.890376 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:40.890654 (+ 278us) server_negotiation.cc:407] Connection header received
0811 05:17:40.890686 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:40.890687 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:40.890697 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:40.890775 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:17:40.890953 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:40.891682 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:40.892725 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:40.893068 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:40.893127 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:40.893317 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:17:40.893356 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:17:40.893410 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:40.893485 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 05:17:40.893512 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 05:17:40.894174 5862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:40.890271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41932 (local address 127.28.143.190:33785)
0811 05:17:40.890382 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:40.890384 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:40.890472 (+ 88us) server_negotiation.cc:407] Connection header received
0811 05:17:40.890548 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:40.890549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:40.890615 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:40.890701 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:17:40.890827 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:40.891635 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:40.892797 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:40.893007 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:40.893081 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:40.893281 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:17:40.893423 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:17:40.893479 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:40.893980 (+ 501us) server_negotiation.cc:299] Negotiation successful
0811 05:17:40.894021 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":64,"threads_started":1}
I20260811 05:17:40.896147 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 5199
I20260811 05:17:40.913429 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 5343
I20260811 05:17:40.924882 29246 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY_TM7_/build/release/bin/kudu with pid 5472
2026-08-11T05:17:40Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3057 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6216 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (301885 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST